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The application of the mathematical methods expounded is illus- 
trated by many practical examples related to various phys cal and engineering pro- 


PREFACE 


At present a wide range of problems related to various branches of physics 
and engineering are solved by probability methods. Naturally, no detailed exposition 
of all the multitudinous applications of probability theory and mathematical statistics 
to various physical and engineering problems can be included within the scope of 
one book. For the small monograph hereby presented to the reader a much smaller 
range was chosen: the theory of electrical and mechanical systems with lumped 
constants under a statistically characterized external force. 


The material is arranged in the following manner. After the first chapter 
which is of introductory nature, the principles of the theory of random functions and 
their linear transformations are given in brief. This mathematical material, sub- 
sequently required, constitutes the second chapter. 


On the basis of relationships obtained in Chapter Two, the third chapter deals 
with the general methods of analysis of random processes in linear systems with 
lumped constants. Beside the generally accepted spectral method, the method of 
stochastic differential equations and the method of impulse characteristics are des- 
cribed here in sufficient detail. 


The equations given in this chapter for each of the three methods mentioned 
are applicable not only to systems with one input and one output, but also to the 
most general problem of random action upon æ linear system which has several in- 
puts and outputs. Several problems are then examined by way of illustration. 
These problems are solved by each of the three methods, thus affording their 
comparison. 


In Chapter Four the application of the methods of Chapter Three to a wider 
range of problems is shown (the Brownian movement, electric fluctuations, ther- 
mal movement of a galvanometer, transmission of telegraph signals and fluctuation 
noise througha low-pass filter, elements of the theory of optimal systems). The 
results obtained here should in no case be considered as exhaustive. The author 
has only desired to show with the aid of these examples the diversity of possible 
problems relating to random action upon a linear system and to further exemplify 
the general methods. 


Chapter Five contains mainly the theory of inertialess nonlinear systems under 
random action,for which such general relationships have been obtained which make 
it possible to carry out calculations in all cases metwithinpractice. The hitherto 
litdle-developed theory of random processes in inertial nonlinear systems is briefly 
surveyed in a separate section. 


In Chapter Sixa series of practical examples is given which illustrate the meth- 
ods of Chapter Five. 


In the course of writing this book, the author has endeavored to confine him- 
self as far as possible to the higher mathematics course given in technical institutes. 
Only elementary notions of the theory of probability and operational calculus are 


required of the reader. These notions can, if necessary, be easily obtained from 
existing specialized literature. The author has consciously allowed some instances 
of lack of mathematical rigor in computations. Refraining from excessive mathe- 
matical rigor makes it possible to substantially simplify the exposition and, at the 
same time, in the overwhelming majority of practical cases, it does not impose any 
restriction on the applicability of results obtained. The author assumes that some 
readers will easily notice and supply these deficiencies in rigor while others will 
readily accept them. 


In conclusion the author considers it his pleasant du:y to express his gratitude 
to the critics Prof. S.M. Rytov and Dr. of Phys. Math. Sciences Yaglom for a 
number of valuable critical remarks, and the editor,Candidate Phys. Math. Sc. 
A.I. Kostienko,for careful preparation of manuscripts for the press. 


Chapter One 


GENERAL CONCEPTS OF RANDOM PROCESSES 


§1. Dynamical and Statistical Laws 


Any phenomenon occuring in nature is bound up with an infinite set of 
other phenomena. Choosing any such phenomenon as subject of study, we find that 
among these connections there are essential ones, which determine the basic 
features of the phenomenon under study, but there are also nonessential ones, which affect 
only some secondary features. In studying the phenomenon it is necessary to find 
out and take into account all the essential connections and simultaneously to disre- 
gard nonessential details caused by subsidiary connections, Thus, not the very phe- 
nomenon in all its complexity is subject to analysis, but a simplified model of it, 
whose behavior coincides basically with the behavior of the subject of our study in 
all but minor and nonessential details. 


The study of the model constructed leads to the setting up of some laws. Only 
the abstract model of the phenomenon follows these laws. However, if the schema- 
tization of the latter has been carried out properly, the laws also describe 
the basic features of the phenomenon studied. Thus the criterion of correctness of 
a model adopted in some theory is the agreement between theoretical results and 
practical, experimental data. 


The model of the phenomenon, later to be analyzed, should be con- 
structed on the basis of making explicit its connections with other phenomena. Here 
the subdivision of factors into essential and nonessential ones depends notonlyon 
the specific nature of the phenomenon itself, but also on the problems that the 
theory will have to solve. 


At present a great many schemes of physical phenomena are known, in which 
for definite external forces, the system's behavior is fully determined by its 
initial state. Such are, for instance, the free fall of a body in a gravitational 
field; the two- and many-body problem well known in celestial mechanics; an 
electric circuit with constant parameters under given excitation, etc, The 
laws which apply to schemes of this sort are known as dynamical. These laws are 
characteristic of a unique specification of the consequences of a given causes. 


Beside models of phenomena which lead to the setting up of dynamical 
laws, other well-known models lead to the formulation of laws of a different 
nature -- statistical laws. To clarify this concept, let us consider as an example 
the model employed in the kinetic theory of gases. 


The kinetic theory of gases deals with such parameters of the gas as pressure, 
temperature, viscosity, specific heat and others, These parameters characterize 
the gas as a whole and are determined by the combined action of all its molecules, 

A gas in an assembly of a great many molecules. In collective phenomena of 
thermal motion of a gas, the individual features of the behavior of its separate 


particles are obliterated and the parameters mentioned are mainly of a statis- 
tical nature, i.e., are obtained as a result of averaging the effects of the indi- 
vidual particles. Therefore, the kinetic theory of gases can be constructed only 
on the basis of a statistical model of a gas, a model that a:lows to formulate suitable 
statistical laws. 


At the basis of the classical kinetic theory of gases lies the following model. 
A vessel of.arbitrary shape contains a given number of gas molecules 
of definite mass. Each molecule is regarded as an ent.rely free body 
not acted upon by gravity and by other molecules (otherwise than by colli- 
sion with them). In the interval between collisions wit: other molecules or with 
the walls of the vessel each molecule moves ina straight line. The change of di- 
rection in collisions follows the laws of collision of elastic spheres. The initial 
state of the molecules is statistically characterized: thelr root-mean-square 
velocity is given and it is assumed that each molecule can with equal probability 
be at any point in the space enclosed by the vessel and have any direction of the 
velocity vector. 


Side by side with the described statistical model of a gas, a dynamical model 
of it can also be constructed. For this it is necessary to consider a vessel of a 
given shape instead of an arbitrary one and also to indicate the positions and velo- 
city vectors of all molecules at a given initial time momert. 


Provided that we have at our disposal unlimited conputational means, 
we can apply the laws of mechanics to the above-menti«ned dynamical model 
and compute the trajectory of each molecule for a time interval of arbitrary du- 
ration. However, the computational difficulties connected with the solution of such 
a problem are practically insurmountable. This becomes obvious if we consider 
that one cubic centimeter of gas at normal pressure and temperature 0°C contains 


approximately 3 - 1019 molecules, the number of collisions of each being of 


the order of 109 per second. The impossibility of carryin.: out the necessary com- 
putational work shows the infeasibility of the dynamical riodel. 


Yet another, more serious deficiency of fundamental nature affects 
the dynamical model in this problem. Actually the beh: vior of an individual 
molecule does not make it possible to draw any conclusion: about the properties of 
the gas as a whole, whilst these are the very properties w th which the kinetic theory 
is concerned. Consequently, in this case the dynamical nodel is in principle 
unsuitable for establishing the laws in which we are int=>rested. 


In the case examined, the statistical approach is nec2ssitated by the fact that 
a very large number of particles participate in the phenom2non. We shall show an- 
other example in whichthe statisticaltreatment is expedient, but where the collective 
character of the phenomenon is somewhat different. This >xample refers to radio- 
reception in the presence of random interference. 


If the form of the interference acting at the time wh2n useful signals are 
transmitted is known, then, knowing the construction and parameters of the radio- 
receiver, one can always calculate with more or less labo: the distortion of the use- 
ful signals by the interference. However, the results thus obtained are of no essen- 
tial value for the theory of radio reception, since they are related to the particular 
case at hand and do not make it possible to draw genera. conclusions about the 
effectiveness of any given radio-receiving apparatus, 


In contrast, the statistical approach to the proble n makes it possible to 
establish what distortions appear on the average (e.g., the average number of dis- 
torted telegraph signs) for a given interference level. This makes it possible to 


get an idea of the quality of the radio-reception method applied. Here the collective 
character of the phenomenon lies in that the interference effects are investi- 
gated for recurrent reception conditions, which on the average show no varia- 
tion. 


The following conclusions can be drawn from the foregoing. If the course of 
the physical phenomenon is mainly determined by a small number of principal cau- 
ses (necessary events), thenthe dynamical model is the one most suitable for its 
study. Besides necessary events, we encounter other physical phenomena whose 
basic features are determined by an exceedingly great number of factors which are 
on the average of approximately equal effect (random events). We stress that a 
random effect is just as causally determined as a certain event, but it differs 
from the latter in the character of its causes. 


It is impossible to construct a dynamical model of a random phenomenon, 
to establish the laws which govern the individual random event, or to make an 
even rough prediction of its course. 


The laws of random phenomena are revealed by the observation ofa great 
number of events taking place under similar conditions or by their multiple 
recurrence. These laws are qualitatively different from the laws which govern the 
individual event. They are statisticalandare studied with the aid of probability models 
of these events. 


§2. Random Processes 


The random events studied by the classical theory of probability are events 
which can either occur or not occur when a certain complex of conditions is realized. 
Such are, for instance, the obtaining of a given number of points in casting dice, the 
emission ofa given elementary particle in a given time interval by a radioactive atom, 
the distortion of an individual telegraph sign by the random interference in the 
communication channel, etc. The development of physics and engineering has 
made it necessary to study phenomena of a different type-the random events 
continuing in time, or, in other words, random processes. 


One of the first random processes studied by physicists was the Brownian move- 
ment, i.e., the movement of minute particles suspended in a liquid or gas, discover- 
ed in the year 1827 by the English botanist Brown, At the beginning of the twentieth 
century, simultaneously with the elaboration of the theory of the Brownian move- 
ment, the study of random voltages and currents in electric circuits resulting from 
thermal agitation in their elements was begun. In the nineteen-twenties these in- 
vestigations were extended to circuits with electronic tubes. 


In all the examples mentioned, the random process was the result of thermal 
movement of matter. However, it is easy to indicate a great number of random pro- 
cesses caused by quite different factors. Such are, for instance, earth displace- 
ments in quakes, ships rolling on rough sea, vibration of vehicles in motion on an 
uneven road, acoustic noise, time variation of meteorological factors, variations 
in the load on a mains network supplying many consumers, etc. 


The first steps in the theory of random processes, dating back to the begin- 
ning of the twentieth century,are typical of the fact that for each problem encountered 
a specific method of solution was evolved, suitable only for the given problem or a 
narrow circle of related problems. The general theory of random processes ap- 
peared much later. Its foundations were laid by the works of the Soviet mathema- 
ticians A.M. Kolmogorov and A. Ya. Khinchin published in the nineteen-thirties. 


The problems dealt with by the theory of random pro: esses can be subdivided 
into two major groups. The first group contains the proble ms connected with the 
mechanism by which the random processes are produced. Although some general 
methods of investigating the problems of this group can be indicated, the 
specific physical content of each problem generally shows through. 


The second group contains problems relating to randcm influence upon sys- 
tems of various physical nature. Here dynamic analogies exist which make it pos- 
sible to describe phenomena widely divergent in their nature, (mechanical, electri- 
cal, acoustic phenomena) withthe aidof a standard mathematical apparatus. These 
analogies make this fieldof the theory of random processes similar in character to 
the theory of oscillations which studies oscillatory process::s in various branches 
of physics and engineering from a common point of view. 


The present work examines problems of the second group only, i.e., the 
behavior of various systems under given random influences is studied. The mec- 
anism by which these influences are produced is not examined in detail. 


§3. The Role of the Theory of Random Processes in Engineering 


The installations, apparatus and devices of modern engineering are systems 
functioning under some external forces. This applies equally totheir assemblies 
and elements, Examples are: a bridge subjected to the lozd of vehicles and pedes- 
trians, apparatus under electric tension, the rope of liftin ; gear subjected to 
strain, and many others. In accordance with §1 the analysis of phenomena in the 
systems mentioned is preceded by schematization of the objectofstudy. This sche- 
matization should be applied to the properties of the system and to the properties 
of the external influence [input]. 


The methods of constructing schematized models of rral systems are outside 
the scope of the problems dealt with in this book. We shall, therefore, dwell only 
on methods of schematizing external forces. 


The majority of engineering apparatus is intended for repeated use under si- 
milar working conditions, Therefore, their behavior is stu lied under external 
influences of a known character, determined by these conditions. 


Similar though they might be, operating conditions ¿re never identical, 
The same can be said of the external forces. The problem of schematization 
lies in that, that all essential properties and peculiartities of the input should 
be considered while disregarding all ite secondary features. Depending on the 
character of the input, dynamic schematization is more convenient in some 
cases, and statistical in others, 


Dynamic schematization of the input is its representat on by some well- 
defined function of the time, whose form is established by analyzing the work- 
ing conditions. Such idealization is expedient when the forrı of the input varies 
only inconsiderably from case to case. Let us give one :xample. The alternat- 
ing voltage of the mains network, to which the device investigated is connected 
does not have a strictly constant amplitude, frequency and ferm. However, in most 
cases, this voltage can be successfuly idealized, and be considered sinusoidal, having 
well-defined amplitude and frequency. 


It is rational to use the statistical schematization of an input, i.e., to 
consider it as if it were some random process, for which th: investigation of the 
operating conditions gives only probability characteristics, when a great variety 
of essentially different forms of input occurs. 


Some such inputs are readily perceived in the examples of the preceding sec- 
tion, We shall supplement them by one additional example, where dynamic schema- 
tization of the input is possible, but the statistical one is more appropriate. 


Until recently, in communication engineering, when analyzing some apparatus 
or its individual elements, the signals transmitted along the channel were regarded 
as determinate time functions of a certain form (dynamic schematization). Such an 
approach to the problem is only the first approximation to reality. The point is that 
any definite time function transmitted through the channel in a given interval consti- 
tutes but one of the possible variants of the signal. The communication apparatus 
should be designed for all the possible variants of a signal. 


In most cases the number of possible variants of the signal is so great that 
any attempt at a simultaneous or successive consideration of all of them is exceed- 
ingly difficult. The situation becomes even more aggravated by the fact that noise 
of extremely irregular character is superimposed upon the signal in the communica- 
tion channel. Hence follows naturally the conclusion that it is more expedient to re- 
gard the signal transmitted through the communication channel as a random process, 
i.e., to apply a statistical schematization. 


The examination of the external forces on various engineering apparatus shows 
that in many cases the statistical schematization represents more closely their pro- 
perties than does dynamic schematization. The same applies to many problems of 
the theory of communication and automatic control, the theory of recording devices, 
the theory of vibration, shocks and rolling experienced by structures, vehicles, 
vessels and airplanes, and to many other cases. The statistical approach to such 
problems is not yet widely used, mainly owing to its comparative novelty and to the 
related fact that the statistical properties of various random actions are not well 
known. This latter fact has also influenced the contents of this monograph, limiting 
the variety of specific technical examples analyzed in it. 


Chapter Two 


RANDOM FUNCTIONS AND THEIR LINEAR TRANSFORMATIONS 


$4. The Concept of Random Function 


The mathematical image of a physical random process is the random func- 
tion. Our exposition of the theory of random processes in linear physical systems 
is therefore headed by the present chapter, in which we state the basic properties 
of random functions and their linear transformations. 


The random process gives in each of a great number of experiments, carried 
out under similar conditions, a time function f,(t), where k is the number of the 
corresponding experiment. In the course of an individual experiment, a well-de- 
fined value of fk(t) corresponds to each value of the argument t. Thus f,(t) is a de- 
terminate function. It is called the realization of the randcm process f(t) in the k-th 
experiment, The random nature of the process is manifested by the fact that the 
form of the function f,(t) varies at random from experimen: to experiment. To 
characterize the random process it is necessary to indicate all its possible realiza- 
tions and their probabilities, 


In ihe mathematical theory of random processes the following more rigorous 
formulation of the aforesaid follows from the concepts developed by A. N. Kolmogorov 
/1/ in his axiomatic construction of the theory of probability: the random function 
f(t) is defined if a probability measure is given on the set o: its realizations. 


Thus, the random function is a function of two variab-es: the time t and the 
parameter k, enumerating all the possible realizations. For any particular value 
of k the function f(t) is a determinate function. In contras:, ifthe time t is fixed, 
the function becomes a random variable. 


Let us note that among the possible values of t there :nay be such particular 
values for which the random function equals a constant. Fcr instance, for the ran- 
dom function which represents the response of some systen. to a random input, such 
a particular value may be initial moment of this input. 


We shall examine a somewhat different method of defining the random function 
that we shall use below. This method is due to E. E. Slutskii /2/, It can be shown 
that this definition is included in the former as a particular case. 


The random variable f = f(t) corresponding to a selected time instant t is fully 
defined if its distribution function or probability density w(t is given. To character- 
ize the random function, the mentioned distribution function or probability density 
should be given for any time instant which lies within the tirne limits of observation 
of the random process. 


This characterization of the random function is still incomplete. The random 
variables fi z f(t), fy a fita) P fa 2 f(t) which correspond to the various time 


instants ty to: ar ty are in the general case statistically dependent. The presence 


of this statistical dependence makes it necessary to specify an n-dimensional distri- 
bution law w f xs fp vih f) Here the number n has an arbitrary value. For 
each n the time instants ty X to. weka ta can be arbitrarily chosen within the limits 
of observation time. 


Thus, the random function is defined, if for any value of n an n-dimensional 
distribution law w G fo. oan fa) is given, where the time instants t to. sea ta 


can be arbitrarily distributed within the limits of the time of observation. 


In many cases this complete characterization of the random function is super- 
fluous. The calculations necessary for practical purposes can be often carried out 
if only the two-dimensional probability density w f fa) is known. It is sufficient 


for the solution of some problems to know only the one-dimensional probability den- 
sity w(f). This is the case, for instance, if it is required to find the probability 
that f shall exceed a given value a. 


$5. Stationary and Nonstationary Random Functions 


Let the n-dimensional distribution of the random function be given. We shall 
fix the mutual positions of the corresponding n points ty to. anes t on the time axis 


and then displace the set of these points along the time axis without changing their 
relative positions. If all the distributions which determine the random function 
remain constant under this displacement, the random function is called stationary. 
Thus, a stationary random function is statistically invariant with respect to time- 
translation. 


For a stationary random function the one-dimensional probability distribution 
is independent of the time t, the two dimensional distribution is dependent on the 
difference to t only, etc. 

In the general case the said time-homogeneity does not hold, and the random 
function and random process described by it are nonstationary. The multi-dimen- 
sional probability density of a nonstationary random function depends on the posi- 
tion of each of the n time instants. 


The random functions met with in many physical and engineering problems 
can be considered, with sufficient accuracy, as stationary. Stationary processes 
therefore occupy an important place in the modern theory of random processes. In 
a number of cases, however, the random process is essentially nonstationary. 
This hinders us from limiting the theory to the examination of only stationary pro- 
cesses. 


$6. Moments 


As known from probability theory, the full characterization of a random vari- 
able is given by its distribution law. Sometimes, however, such exhaustive informa- 
tion on the random variable is superfluous. In this case the values of the first few 
moments of the distribution law are often indicated instead of the distribution law 
itself. An analogous situation exists in the theory of random processes. Here 


one often refrains from considering n-dimensional distribution laws and operates 
with a finite number of moments. 


The description of a random process with the aid of moments is less compre- 
hensive than in using suitable distribution laws. However, in many problemns of the 
theory of random processes probability densities are rather difficult to obtain, 
while moments are computed by quite elementary means and describé suffi- 
ciently well the phenomenon studied. Moments are therefore widely used in 
the theory of random processes. 


Moments are obtained by the averaging operation. In «onnection with this we 
shall first examine the question of averaging random functior:s. 


As was shown in §4 the value of a random function depends on time and the 
parameter k (realization number), On of the possible methods of computing an ave- 
rage is by fixing a definite time instant and computing the average of the totality of 
realizations of the random function. Here the operation of averaging can be applied 
not only to’ the values of the random function, but to any function of it. Exactly in 
the same way one can select several time instants and apply the averaging operation 
to any function of the corresponding values of the random function, 


Another possibility is: selecting a certain number of realizations of the ran- 
dom function and calculating their time-averages. As in the preceding case the 
averaging operation can be applied to the value of the random function, to any func- 
tion of this value or of several such values at once, In the letter case it is first 
necessary to fix definite positions of the corresponding time for all time-axis points, 
and to displace (in the course of calculation) these points aloig the time axis, with- 
out changing their relative positions. 


Consequently, a random function has average values w‘th respect to the set 
of its realizations, and time-average values. The first corr2spond to definite time 
instants, the second--to a definite realization of the random :unction. 


In the general case, when the random process is nonstationary, the only aver- 
age values of the first kind are of interest. These values are determined by the set 
of possible realizations of the random function and characterize the latter as a whole. 
For a stationary process the time-average values are also considered. 


We now proceed to the definition of moments, starting with a one-dimensional 
distribution w(f) at a time instant t. The moment of order v of this distribution is 
defined by: 


m, (t) = f romar (v=1, 2, 3, ...). (2.1) 


If follows from expression (2,1) that the moment m,(t) is the average of the 


random variable f¥(t) over the set of realizations of the random function. This 
average is called the mathematical expectation of the random function fY (t) at time 
t. We shall henceforth denote the mathematical expectation ef a random quantity 
A by the symbol M[A]. Accordingly 


mt) = Mf (Oh (2.2) 


In general, the moments (2.2) are time-dependent and are constants for 
stationary random functions. 


10 


The simplest moment is the moment of first order or first moment 
m(O=M(SOl (2,3) 
which is the average of the random function at the time t over its set of reali- 
zations. 


The second moment 
m, (t) = MIf? (t)| (2.4) 


is the average of the square of the random function at the time t over its set 
of realizations. 


For the solution of numerous practical problems it is often sufficient to con- 
sider only the first two moments of the random function. In some cases sucha 
characterization of the random process is insufficient and it is necessary to have 
recourse to moments of a higher order, i.e., the third and fourth. 


The moments of multi-dimensional distributions are introduced in an analo- 
gous way. The moment of order v of the two-dimensional distribution wt, f,) 


will be any function of the form 


migli 2) = ff fifo. fal df, df, = MIS -Al (2.5) 


-0 


where 


i+jy=u A=) h= ft. 


It is easy to see that there are altogether v —1 different moments of a two- 
dimensional distribution function, these moments corresponding toi = 1, 2, 3,.., 
v —1. Fori =0 ori =v the moment ofatwo-dimensional distribution degenerates 
into the corresponding function of a one-dimensional distribution. 


The simplest moment of a two-dimensional distribution is the mixed second 
order moment 


My (ti, t) = MIS) Sta. (2.6) 


The moment (2.6) is the average over the set of realizations of the 
product of two values of the random function corresponding to the instants t and 


tz If the moment m (t,, ta) is given, this defines the second moment (2,4) of the 
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one-dimensional distribution, since, setting t = to = t in (2.6) we have 


mit, = M [f = m (0. (2.7) 
For a stationary random function of the moments (2.5) are functions of the 
two variables ti and tz For a stationary function a time-translation leave the 


probabilities of the process characteristics unaltered and hence the moments 
are functions of only the time interval += [|t —t,|. 


f) 


Any moment of order v of the three-dimensional distribution wli fy 3 


is expressed by: 
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mijl be td = M| ED PD A th, (2.8) 


where 


i+j+k=y. 


We shall note the fact that stating the third order moment of the three- 
dimensional distribution 


Mari (tre tz ty) = MUSE): SG) SGD (2.9) 


defines also third-order moments ofa two-dimensional distribution. The transition 
to the latter is effected by equating the corresponding two time instants in expres- 
sion (2.9). In exactly the same manner, fourth-order moments of a two-dimensional 
distribution are particular cases of fourth-order momentt of a four-dimensional 
distribution. 


We shall draw the following general inference from the aforesaid. A random 
function is defined, i.e., an n-dimensional distribution law wif, fo k tè is given 


for any n, if for any y the moment 


Milby ta... t) = MUS ED S). SN. (2.10) 


v 
is known. 


By equating appropriate arguments to one another we can obtain from the 
moments (2.10) all other moments, whilst the totality of ııı moments determines 
uniquely the n-dimensional distribution law for any n. 


Since we shall chiefly meet the following moments of the type of (2.10), we 
shall denote for brevity: 


ma. allo ta o L=M, (b ta o E) 


v 


(2.11) 


Beside the moments examined above one often mee:s also central moments. 
A central moment of order v of a v -dimensional distribution is defined by: 


Bs llo to wees j= MII) — m, (t,)} (S(t — m, (h) tee 
wo (A4) -m C). (2.12) 


It follows from (2,12) that for a random functior whose first moment 
identically vanishes, every moment is central. 


A central moment of the first order always equals zero. In fact, since the 
mean value of a sum equals the sum of mean values of tre summands, we have, 


1) = MISO — m (A = MIN — m t) = 0. (2.13) 


We shall examine a central moment of the second order. Carrying out cal- 
culations analogous to the preceding we obtain: 
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Hoty, te) = MIF (4) — m, (4) (SG) — m, OY = 
= MIS (4) SDA — m (hy) MIS) — m, t) MIAN + 
H+ my (A) «my (f2) = Ma (ty ta) — my (fy) > my (ta). (2.14) 


The relationships for central moments of higher order are obtained in an 
analogous manner. 


In the particular case when t5 t,=t, expression (2.14) gives 


2 
pat) = M [1A 0 — m, (DV) = m, (0) — mi (0. (2.15) 
The quantity Ho (t) is the average over the set of realizations of the square 


deviation of the random function from its average value m (t) corresponding to a 


time instant t, i.e., is equal to its dispersion [variance] ortt) at this instant. 


The second order central moment Ho te to) is often called autocorrelation 
ratio and denoted by k(t,, ty). Thus, 
k(t be) = Pelt. 4). (2.16) 
The expression 
R(t. ta) 
bbe 4) = Sa AG (2.17) 


is called the normalized autocorrelation ratio, It corresponds to the correlation 
coefficient widely used in probability theory. 


For obtaining the above-examined moments, the operation of averaging over 
the set of realizations of the corresponding random function was employed. This 
method is general and equally applicable to stationary and nonstationary ran- 
dom functions, For studying random functions another method of determining 
moments, giving the same results, is also possible. 


The statistical properties of a stationary random function do not vary with 
time. It follows from this time-homogeneity that the average over the set of 
realizations calculated for a chosen time instant gives the same result as the 
time-average of a single realization. In the latter case the time of observation of 
the random process should be sufficiently long (strictly speaking, infinite), This 
proposition is known as the ergodic theorem. 


The ergodic theorem is valid under sufficiently general conditions which are 
generally fulfilled in practical problems. The proof of this theorem is given in the 
work of A.M. Yaglom /3/ as well as in the monograph of V.I. Bunimovich /4/. 


Thus, it is possible to calculate the moment of a stationary random process 
when a single realization of it f(t). corresponding to a sufficiently long observation 


time, is given. The computation of, say, the first moment is carried out thus: 


T 
m = MI (N = lim + f tna. (2.18) 
> oOo $ 
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For the second moment of a two-dimensional distribution we have: 
T 
N t : 
me) = MIN LO = lim > f HDfeltrorde. 219) (2.19) 
> oo 
o 


The computation of moments from a single realization of the observed ran- 
dom process has great practical advantages. 


§7. The Derivative of a Random Function 


Let us consider the derivative of an individual realization f(t) of a random 
function f(t). Since the realization f(t) is a determinate function of time, an analogous 
statement with respect to its derivative f ft) is also true. Thus, the derivative 
f(t) is a derivative in the usual sense and all the propositions of the differential 


calculus of determinate functions are applicable to it. 


Let us now fix some time instant t and pass fron: one realization of the 
random function to another. In this case the value of the Jerivative f'y,. (t) will 


vary in a random manner with the realization number. This random variable is 
called the derivative of the random function f(t) at the moment t. The following 

definition is equivalent to the above: the derivative of the random function at the 
moment t is the limit* 


r 
oe at dt fo (2.20) 
We shall explain the relation between the random function itself and its 
derivative. For this we shall first take a normal distritution function, i.e., a 
function defined by normal n-dimensional distribution laws. We shall assume for 
the sake of simplicity that the mentioned function is stationary. Its two-dimensional 
probability density is then of the form 


a l 2-2 
w = yey exp | — EA 1 (2.21) 
where we denote for the sake of brevity 

h=) h= Sta. (2.22) 


The parameters o 2 and p which appear in (2.21) are, respectively, 
the dispersion and the normalized autocorrelation ratio o: the random process con- 
sidered, the first one being constant with respect to time by virtue of the stationary 
nature of the process, and the second one depending only on the time interval 
At=t,-—-t,. 
2 1 


We shall change the variables in the distribution law (2.21) thus 


LSF h=sft+4f. (2.23) 


* The conditions for the existence of the limit (2.20) will be considered below. 
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Hence we shall obtain after simple transformations* 


rae l PUASA- + af? 
wl Sse [ee |. 22% 


In order to obtain the one-dimensional probability density for the incre- 
ment åf it is necessary to compute the integral 
400 


w (Af) = Jev. Af)df. (2.25) 


-0 


The value of the following definite integral is given in handbooks (see /6/) as 


+% q 
f eE dx= er ⁄ z, (2.26) 
-% 


Making use of (2.26) we find: 


1 Af? 
eee th ne pee ARE (2.27) 
Vin ViVi; exp | 2(V B71 a 


Thus the increment Af represents a normal variable with dispersion 


w(Af)= 


oap = 237(1 — p). (2.28) 


Consequently, if the limit (2.20) exists, then, passing to the limit for 


* As known from probability theory /5/, the rule of the change of variables as 
applied to two-dimensional distribution laws is formulated in the following way. 
If the new random variables f and ņ are related to the old variables x and y 
by 


E=f(x,y, qn=g (x, y), 


then the two-dimensional probability density w(Ẹ, ņ) is expressed by the two- 
dimensional probability density w(x, y) in the following fashion 


w(x, y) 
ORUE 
ð (x, y) 


w ($, n= 





where 


a at | 
ok n ox oy) 
dix, y) | òn On 


ax Oy 
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At+0, we shall obtain again a normal variable. Thus, tie derivative of a norm- 
ally distributed random function has also normal distribution. 


Let us find the dispersion of the derivative. The dispersion of the quotient 
Af Atequals 


a 2o7[1 — p (Ad) 
aspis = 4 ee. (2,29) 





For Af+0 expression (2.29) becomes indeterminate since p(A4t)—+ 1. To re- 
move this indeterminacy we shall expand p(Af) in a power series in terms of Af: 


p(t) = 1+ p7(0)- At + 5 0"(0)- Art... (2.30) 


Expression (2.29) assumes now the form: 


of (0) MF ay TAONE: is 


aby, at = — 222 —__—,, ———.. (2.31) 


We shall examine this result. By virtueofthe fact that the autocorrelation 
ratio p(Af) is an even function, two cases are possible: 1) the point Af = 0 is an 
extremum, i.e., p’(0)= 0; 2) the derivative p’(0) at the point ât = 0 does not 


exist (such an example is the function p(4f)— e-*!4f!). 1: is obvious that the deri- 
vative of a random function exists only in the first case. For »’(0) = 0, going 
over to the limit in (2.31) (Af +0) we have: 


M (4) = — 2%” (0). (2.32) 


Expression (2.32), if meaningful, is always positive. Consequently, for 
p’(0)—-0, the function p(\f) has an extremum at the peint A = 0, this ex- 
tremum being a maximum, since #(0) = 1 and for Af =Q we have p(dt)< 1, 
whence it follows that we always have p” (0)<.0. 


We shall now sum up the results of the foregoing analysis. As mentioned 
above, the derivative of a normally distributed random function, if its exists, has 
normal distribution. This result is directly related to the statement of probability 
theory that any linear function of several normal randcm variables has also a 
normal distribution. As the structure of (2.20) shows, the process of calculat- 
ing the derivative is equivalent to forming a linear fun:tion of the two random 
variables f(¢+ At) and f(t) with subsequent passage to the limit. 


It is necessary for the existence of the derivative that there should be a statis- 
tical dependence between values of the random function which are sufficiently 
near in time. But if values arbitrarily near in time are ‘ully independent statis- 
tically, the probability that any realization of the random function is discontin- 
uous at all points of the time axis and does not have a derivative is unity, i.e., the 
random function does not have a derivative. However, the existence of a statis- 
tical] dependence is not sufficient. The statistical dependence shouldbe such that the 
cendition p’(0) = 0 is fulfilled. 


If we restrict ourselves to the examination of randon variables of finite aver- 
age values and dispersion, which case is of fundamental i iterest in practice, we can 
sav that the sum of several variable having the same distribution law also obeys 
this law only in case the latter have normal distribution. No other distri- 
bution law has this property /7/. Hence it follows chat under the condi- 
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tions stated differentiation does not affect the distribution law only for random 
functions of normal distribution. 


The above reasoning is readily generalized for derivatives of higher order. 


Let us find the law by which moments are transformed in differentiation. We 
shall first consider the first-order momentof the derivative. Since in all cases 
the operations of forming the average and of passage tothe limitare interchangeable, 
we can write: 


df) f+ 3N—f (9) 
Ma) = n] = 
MIS(E+ SI MISCO] _ dm 4 


= lim (2.33) 


at>o “ 





Consequently, the first-order momentalso undergoes differentiation when the 
random function is differentiated. It is obvious that the first-order moment of 
the derivative of a stationary random function equals zero. 


Generalizing the result obtained for the case of the first derivative, we have: 








df hee d”m, ít) : 
“m [ae ]= ar (2:39) 


Let us proceed to the second-order moment of a two-dimensional distribution 


of the derivative, Since passing to the limit and averaging are interchangeable 
we can write 


am [2 af tt) 


dt, dt, 
Sle t+ 84) —S (HY SF (ly + 46) — F(t) 
= li fli + ON) — FAY Ste + dt) — Fd) (2.35) 
sso. [ Bh Be ] 
Phd 


Introducing the moment m(t. ta) we shall give expression (2.35) the form 


[spun , 


dty dh 
= lim ~ [= (4, + bli la + bl) — melh ha HAt O 
at > 0 At, A; 
SEEN (ty + ty, te) — my (ty, ty) 
Mlh 1: fe) — L 


The following final result is obtained from (2.36): 


dft) af (ty) ] __ Pms (ty l3) 
[ dt, | dt, | anoh (2.37) 

Thus, the second-order moment of a random function is differentiated 
twice when the function is differentiated once. Formula (2.37) is readily general- 
ized for the case of a moment order v: 


MAL Af) Af] mt ty td (2.38) 





“dt, d dt, JT dtd... 





In precisely the same way it is possible to obtain analogous results for higher 
order derivatives. The moment of order v of the n-th derivative of a random 
function is given by: 





M [LLG PL) FY] o_O ms ly ty) (2.30) 
at" a at otat.. ot  * ` 


For a stationary randon functions differentiation with respect to time can 
be replaced by differentiation with respect to the length of interval separating 
time instants. As an example we shall consider the seconł-order moment of a 
two-dimensional distribution of the derivative. The right-h:.nd side of expression 
(2.37) can be thus represented for a stationary function 


Pm, (ty, t) è ð Omg (2) A 


ddh dha kl A Oh, 





(2.40) 


where 
t=|h—f|- 


Keeping in mind that the derivatives 0</dt, and 0:/df, equal unity in their abso- 
lute value and are of opposite sign, we obtain for a stationary random function: 


dt, dh at (2:41) 


If we assume in (2.41) t = 0, then as could be expected we shall obtain a re- 
sult coinciding with (2.32). 


$8. Integral Transformations of Random Fınctions 


An integral transform of a random function f(t) is defined as 
ò 
F=f foewat, (2.42) 
“a 


where ¢(¢) is a determinate function. 


We shall first explain the nature of the quantity F. Ve shall assume that a 
determinate function Ẹ (f) is given and the integration interval (a,b) is fixed. Then opera- 
tion (2.42) gives a certain numerical value of F for every 1ealization of the ran- 
dom function /(/), This value varies in a random manner f:om one realization to 
another i.e., for given ọ (f) and integration limits the integral transform (2.42) 
is a random function of the realization number of the rzndom function f(t). 

The probability function of the random variable F depeniis on the chosen func- 
tion 7(f) and the integration limits a and b. 


For investigating the statistical properties of the random variable F we shall 
break up the integration interval (a, b) into elements Af andgc over from the integral 
(2.42) to the sum 


N 
F=% f (EAN) @ (I At) àt. (2.43) 
al 


As mentioned in the preceding section, the distributicn law of the sum 
does not, in general, coincide with the distribution law of the summands. There- 
fore, in general, the distribution law of the random fun:tion is altered by an 
integral transformation. Without examining this problen in general we shall 
mention two important particular cases of it. 


The quantity F is normally distributed if the random function f(t) which 
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undergoes the integral transformation has normal distribution. The distribu- 
tion of F is normal, foranarbitrary distribution law of the function f(t), any 
values arbitrarily near in time are mutually independent statistically. The last re- 
suit is the consequence of the central limit theorem of the theory of probability. 


Let us proceed to the elucidation of the laws of transformation of moments in 
integral transformations of random functions. We shall begin with the first or- 
a (F) 

ler moment m, 
equals the sum of the averages of summands, we have from (2.43): 


of the integral transform. Since the average of the sum 


N 
mP = Da AN) 9 (Af) At (2.44) 


or,passing to integral notation we obtain 


b 
mP = f mO (1) @ Hdt. (2.45) 
a 


Thus, in integral transformations of a random function, the first-order mo- 
ment undergoes an analogous integral transformation. In the particular case of a 
stationary random function its first-order moment does not depend upon time and 
expression (2.45) becomes simplified thus: 


h 
mP =m? f oat. (2.46) 
a 


Let us calculate the second-order moment of the integral transformation (2.42) 
We have from (2.43): 


N 2 
F? = [ Evwan ewan ar] = 
N N 
£ 2 PRAY At) f (j AL) ẹ (EAL) @ (j Af) AL?. (2.47) 


With respect to an individual term of the sum (2.47), the operation of 
averaging gives: 


M [fi Ot) f (7 St) 9 (iA) g (J Af) (ay = 
= Mf U At) S Cj AND) p (0 AL) g (7 Af) (At)? = 
= mf" (1At, j At) o (i At) @ (j At) (At). (2.48) 


In view of this result we can write: 
N N 
M[F]= mP = 2a mY (At, j At) q (AL) ẹ (j At) (AN (2.49) 
or going over to the corresponding integral notation we obtain: 


6 b 
mP = f f moa. tde UDe (ta) dt, dta (2.50) 
aa 


Comparing this expression with (2.42) we notice that in the integral transforma- 
tion (2.42), the second-order moment of atwo-dimensional distribution of the random 
function f(t) undergoes an analogous double integral transformation. 
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One can readily convince oneself that it is necessary to apply a double integral 
transformation to the correlation function for calculating the dispersion of the inte- 
gral transform F, i.e., 


b b 
op = f f RP, td alte dt, dte (2.51) 


a 


The generalization of the results (2.45) and (2.50) obtained above gives for 
the moment of order v of the transform F the following 2xpression: 


b 

F 
mP f 
a 


—— 
X 


b 
fala, ty oo. Datla)... 
.-. G) dt, dly... dt, (2.52) 


Thus the calculation of the moments of order v of the random variable F is 
carried out by means of a v-tuple integral transformation of the v-th order moment 
of the v-dimensional distribution of the random function f(t’. 


§9. A Set of Several Integral Transformations of the Fandom Function 


We shall examine a set of m different integral traisformations of the 
same random function f(t): 


b 
F, = fo y tdt, 


a 


6 
F=f sO gatos, (2.53) 


ò 
Fn = | S/O 0n O dt, 


where 9,, r...» m are some determinate functions. 


The moments of each of the integral transforms(2,33)can be calculated with 
the aid of the relationships of the preceding section. The random quantities 
F Fo sits, ate Fo are in general statistically dependent anı: the m-order moment 


of the m-dimensional distribution of these quantities is to te calculated. 


Representing each of the integrals (2.53) in the form >f a sum of type (2.43) 
we obtain: 


N 
FF... Fim = Dreane canar} x 


N N 
x PICT) atana) Bg (È 1m) om (lm atyart (2.54) 
or, in a different manner 
N N N 
Fifa... Fa= D 2 os RG AL) Sf (lg AL). fl AD X 
X gı (ty At) P2 (l2 Ôt)... Pom (hy Af) (AL)™. (2.55) 
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Now, carrying out the operation of averaging analogous to that of (2.48) and 
reverting to integral notation, we have: 


MIF, Fy... Faq) = m = 


b ò 
=f... fal (tye tov coos bm) Pi (ta) G2 (ty) » 


» Gm (Ln) db, dl; ... dt m (2.56) 
The result obtained can be readily generalized for different integration limits 
of (2.53). If these limits are correspondingly (a,, bi), (ay, bo), arabes (a bi) one 


can always choose such an interval (a, b) which contains the integration intervals of 
all the integrals. Each of the integrals of (2.53) can then be written as follows: 


dk b 
F= frond =f sOgod. (2.57) 
Ok u 


where for a, <¢ < bk we have 
p (= 9, (A, (2.58) 
and fora Lt < âk or << t<b we assume 


$, (0) = 0. (2.59) 
Now, writing the required result in the form of (2.56) we notice that in each 
integration we can return to the original limits. Thus, 
b, by 
MIF Fy 00. Fe) = aD =ff-. Js mI G ta ce tm) X 
ay ty 


X pilh) Gate)... aia dt, dla... dtm (2.60) 


For any pair of transformations of (2.53)we can write by virtue of (2.60): 


P= MIF;F saf Tao dt. (2.61) 


a ay 


Consequently, to obtain the mixed moment of a set of integral transforms 
it is necessary to apply successively all the transformations, defined for the 
random function itself, to its corresponding moment. 


The result obtained allows for an obvious generalization for the case when 
each of the integral transformations of (2.53) refers to a specific random func- 
tion and all the m random functions are statistically interdependent. 


§10, Linear Transformations of Random Functions 


The present section is concerned with the generalization of relationships 
obtained in preceding sections to the case of arbitrary linear transformations of 
random functions. These results are given in the works of V.S. Pugachev/8, 9/. 


For the sake of clear and brief notation of the forthcoming theorems we shalluse 
operational formalism, We shall denote an operation performed upon a variable or 
function x as the product ofthis variable or function and the corresponding operator, 
e.g., the operator A. Thus, the result of carrying out the operation A upona vari- 
able or function xis symbolically expressed as follows: 
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y = Ax. (2.62) 


The operation A can vary greatly from case to case, It can, for instance, 
determine a function y(x), i.e., indicate a rule according t which the value of the 
variable yis found from the value of x. The integral transformations examined in 
the preceding sections bear a different character, these transformations being 
operators which determine the quantity F from the given function f(t). The transi- 
tion from a function f(t) to another function q(t) carried out in a certain way can also 
constitute such an operation. An example of such an operation is differentiation. 
Finally, it is not difficult to think of a great number of various operations in which 
a function y(t) is determined by the value of a variable x 


From all these possible operations we shall pick out ‘inear operations, i.e., 
Operations which have the property 


A (a,x, + aat ... + antn) = 
= a,Ax, + a,Ax,+ ... +4,Ax,. (2.63) 


Equation (2.63) signifies that the operation A upon a linear function of the 
variables or functions es x is a linear functior. of the results of this 


operation upon each of these variables or functions. An oferator which fulfils re- 
lationship (2.63) is called linear, 


We shall call an arbitrary linear operation carried o1it upon a random func- 
tion a linear transformation of the random function, It .s easy to see that dif- 
ferentiation and integral transformations of the random junction are particular 
cases of its linear transformation. 


We shall examine an arbitrary linear transformation of the random func- 
tion f(t): 
F(z) =A: f(b. (2.64) 


The argument may remain invariant (z = t) in the linear transformation 
(2.64) as, e.g., in differentiation,and may change as was the case with the integral 
transformation, 


If the function undergoing a linear transformation has normal distribution,the 
outcome of the transformation is again a normally distribu ed variable or function. 
For any other distribution law having finite mean and dispersion, the distribu- 
tion law is changed by the linear transformation. 


As to the operations of differentiation and integral transformation, we have 
noted that the n-th moment of the image of f(t) under a linear transformation 
A equals the n-fold iteration of A applied to the n-th moment of ft): 


MPZ Za ooo Zad) = Ar, An o At, MDP (ys tas oe; fn) (2.65) 


In the proof of relationship (2.65) with respect to bot! these operations the 
fact was used that the order in which the operation of diffe: entiation(or integral 
transformation) and the operation of averaging are carried out is immaterial 
(the commutative property of these operations). Since tie operation of averag- 
ing commutes with any linear operation, (2.65) applies to all linear transfor- 
mations of a random function. 


As an example we shall apply result (2.65) to a linea: differential equation of 
n-th order 


22 


Ay = Bx. (2.66) 


where x = x(t) is a known random function, y = y(t)is an unknown random 
function, A, and B, are linear differential operators which written out equal: 





a” dn! d 
A= ân a +4 Ww fees Hag t’ (2.67) 
d” a™-t d 2 68 
By = Dm Fie toma Goma + eo Hbg tie (2.68) 


Equation (2.66) in which the functions x(t) and y(t) are random is called a 
stochastic differential equation. 


In accordance with the results given above, a differential equation analogous 
to (2.66) is valid for the first-order moments m,w and mw: 
Am” (0) = Bum (D). (2.69) 
For mixed second-order moments we have: 
Ar Aum? (ty. t) = Bey Baume (ty, ha), (2.70) 
or, in full 
o” ð” ty) 
a, — awa dy || an — ae aom? (ti. 4) = 
(miit o Hao) ane + o Hao) t 


poa. bo (bn 2 b,\m (ty, ta (2.71) 
= (bq Sot... +0)(Om rot os Hy) mM, ta 
i 2 


The coefficients ay and by in operators (2.67) and (2.68) and, consequently, in 


equations (2.69), (2.70) canbe either constants or determinate time functions. 

Thus for the moments of the first and second order we have obtained: in 
the first case an ordinary differential equation, and in the second case a partial dif- 
ferential equation. It is easy to set up in an analogous way differential equations 
for moments of any order. 

The results given in§§7and 8 have been generalized above for the case of 
arbitrary linear transformations. The results given in §9 are also capable of an 
analogous generalization. 

Let a set of arbitrary linear transformationsofa random function be given: 

F, (2) =A, f (8). 
F, (2) = Af (N). (2.72) 
Fa (2m) = A mf (£). 
Then we have by analogy with equation (2.60) 
F = mh = AA 
MIF,Fz ... Fl = Mm = AA- 
oo Amtn MA (ls fay oee Lm. (2.73) 


When all the transformations of (2.72) contain different random functions 
which are statistically interdependent the result is written in an analogous way. 
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Chapter Three 


RANDOM FORCES UPON A LINEAR SYSTEM WITH LUMPED CONSTANTS 


$ 11. Statement of the Problem and Ter:ninology 


The processes which occur in electrical,mechanica , and electromechanical 
systems are described by similar differential equations. For any system of the 
mentioned three types an equivalent system of any one of hetwoother types can be 
given. This makes it possible to construct general meth: ds of analysis of random 
processes, equally applicable to any of the indicated c.asses. Consequently, 
we shall henceforth not go into the physical method of realizing a system sub- 
ject to random action but shall examine the problem ir its most general for- 
mulation. 


The locus of points where random force is applied to the system is called 
its input. Depending upon the nature of the random process studied, the exter- 
nal force can be of a different physical nature: an elect-ical voltage or current, 

a mechanical force, torque etc. Similarly, the concept cf the"locus inthe system" 
should be made more exact in specific cases according tc the method of realization 
of the system. Such are; a pair of terminals of an electrical system across which some 
electric voltage is applied; a point executing translato-y motion in a me- 
chanical system, at which point the mechanical force is applied; the axis of rotation- 
al motion in a mechanical system when a torque is exerted on this axis. 


The locus of points at which the response to the applied force is observed, is 
called the output of the system. The response of the system may be, for 
instance, a voltage or current in an electrical system, a displacement or angle of 
rotation in a mechanical system. 


The simplest problem is the investigation of a random process in a system 
with one input and one output. It is assumed in this case that a sufficiently complete 
characterization of the input applied to the system is .vailable. The statistic- 
al properties of the system's response are here the sibject of study. It isalso 
important in some cases to study the statistical correlation between response and 
external force. The question, what kind of statistical chiracteristic of the external 
force is sufficient, is solved with regard to the required exactitude of the statistical 
properties of the system's response. 


The general case is that of a system of m inputs anc noutputs. Of interesthere 
are not only the statistical properties of each of the responses separately but also 
the nature of the statistical correlation between the responses at different out- 
puts as well as that between responses and external forces, 


The course of the random process in the system dependas upon its state at 
the initial time when input is applied, i.e., upon initial conditions. The latter may 
be specified uniquely, as is done in the theory of dynamic processes, or statistic- 
ally. Only in a particular (but important) case, where on y the asymptotical be- 
havior of the system for f — œv is studied, is the knowledge of initial conditions 
superfluous. 


24 


$ 12, The Methcd of Stochastic Differential Equations 
Let us consider a random process in a system having one input and one out- 
put. As well known, in a dynamic process the response y(t) of the linear system 
with constant lumped parameters under external force x(t), satisfies an ordinary 
differential equation with constant coefficients: 


Ary (t) = B,x (t). (3.1) 


Here A, and B, are linear differential operators: 


a an-t d 

Ar-= Gn ga t On Gat Ha Gt Qa, (3.2) 
a" aut d 

By = bm Gem + Pm a gine He H Or Gy Oo (3.3) 


Equation (3.1) is equally applicable to the description of a random process in 
the system. Then the external force x(t) is a known random function of time and 
the problem consists in investigating of the statistical properties of the response 
yit) at the output of the system. Equation (3.1) then becomes a stochastic equation. 


As had been already shown in § 10, a transformation of the distribution law 
takes place generally in a linear system. Without discussing the problem of the 
transformation of distribution laws in linear systems we shall turn now to the much 
simpler problem of the transformation of the moments induced thereby. 


Using the results of § 10 which deals with linear stochastic differential equations, 
one can set up differential equations of the usual kind, corresponding to the stochas- 
tic equation (3.1) and which describe the behavior of the moments of the response y(t). 


For first-order moments we have the usual ordinary differential equa- 
tion of n-th order: 


Ayn? (t) = Ben® (t). (3.4) 
Let us examine some particular cases of equation (3.4). The simplest case 


is the one in which the input is stationary, i.e., m,(*) = const. and the first 


moment of the steady-state response, i.e., att -»oo, is required. Then the 
operators A; and By are simplified in the following manner: 

Ap =a, By = bo (3.5) 
and the required first-order moment is: 


ho 
a) ie a ae (3.6) 
If the input is stationary but the process leading to the attainment of a steady 
state is of interest, one has to superpose upon the steady-state solution for 


the first-order moment given by (3.6) a transient process described by the homo- 
geneous differential equation: 


Aym® (2) = 0. (3.7) 
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The solution here obtained of equation (3.4) with constant right-hand side 
should satisfy the initial conditions of the system. 


If the first-order moment of the input varies in time in a known way and 
it is required to obtain a full description of the behavior of the first-order moment of 
the response, the most general form of equation (3.4) should be used. The time 
variation of the first-order moment of the input may be the result of either 
the nonstationary character of the random force itself, r of the superposition 
of a regular input upon the random process. A mixed case, when both of the 
mentioned factors appear, is also possible. 


The solution of equation (3.4) can be effected by any of the known methods. 
Since the random nature of the input does not introduce any new features into 
the solution, we shall not dwell upon its technical aspect. 


Let us turn to second-order moments. The second-order moments of the 
input m,'%*) (t rtp) is, in general, a function of two variibles, as is the second- 
order moment of the response of the system. It was shown in § 10 that the mention- 
ed moments are connected by the differential equation (2.70) whose explicit form 
is given by the expression (2,71), Thus; 


Ay, Arm (ty, ty) = Be, Bumn’ (t £), (3.8) 


where At, A By, » and At,, Bt, are linear differential operators of the form of (3.2) 


and (3.3),where the variable t is replaced by t, and to respectively. 


1 


Equation (3.8) can be solved in different ways, one of which is the following. 


We introduce a new unknown function of the variables t and to 


s(t. = Anm” (t, ty) (3.9) 
and knowing the second-order moment of the input, w find the function 
Z(t, t) = Bm (t, ta). (3.10) 
Equation (3.8) is then transformed into the ordinar y differential equation: 
Ars (tis t3) = Biz (ti. ta). (3.11) 


In this equation the variable t, is the argument and tọ - a parameter. The 
arbitrary constants appearing in its solution are functions of the parameter tz 


After solving equation (3.11) we find the second-or ler moment of the system's 
response from the ordinary differential equation (3.9). in (3.9) the variable ty is 
the argument and t - a parameter. 


Another possible method of solving equation (3.8)is the use of the Laplace 
transform: 


ao 
T= fse dt. (3.12) 
0 
Assuming, for simplicity, zero initial conditions, let us apply the Laplace 


transformation with respect to the variable t, to both gic es of equation (3.8). We 
obtain: 


Ap Aimy (Pr 4) = Bp Bumi? (Pis t). (3.13) 
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where 


Ap, = aap + anM + vee aP, + a, (3,14) 
Bp = bgp + bm pr +... +p + bo. (3.15) 
mi? (Ps =f mP (ty. ye-Ph dh, (3.16) 
mP (Dy. te) = f mP, Gen Me dt, Ga 

0 


The application of a second Laplace transformation with respect to the variable 
tə gives 
2 


ApApmi (Di. M) = BpByms (Pis P) (3.18) 


where Apo and Bye are expressed by formulas (3.14) and (3.15) with p) replaced by 


Pg» and the twice-transformed moments have the form 
ao 
ms (Py. P) = [Pe th) eP at, = 


t kd 
=f fre (lis t) TNFR di, dtz, (3.19) 
o o 


œ 
mP (Pus Pd) = f mY (Pu t) eadh = 
v 


a 


wo 
= f f mP e, terrains, dt. (3. 20) 
o 0 


Thus we obtain from (3.18) for the "double" transform of the required 
moment; 


=- BoB 
MP (Dy, Pr) = GPA mi” Ou Pa) (3.21) 
pA A 
a 
The transition to the function of original variables is carried out for each 
variable in turn and can be achieved by double application of the inverse trans- 
formation or by other methods known from operational calculus /10/. 


Differential equations connecting moments of higher order can be set up and 
solved in an analogous way. 


The method of stochastic differential equations can also be applied to the 
more general problem of a system with m inputs and n outputs. In this case, one 
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has to consider a system of n stochastic differential equations instead of one such 
equation of the form (3.1). These equations are: 


n m 
AY y(t) =D B xA k=l, 2, 0), 
2A yO = BB x ( ) ee 
where ACN and pu) are linear differential operators, it.ndjarenumber-labels of 


the output andthe input, respectively, andka running index enumerating the equations 
of the system. 


The systentf conventional differential equations for the first moments 
At Ș gisk 
zx nD 
Py A mP (9 = 2 5 mP O. (3.23) 


corresponds to the system of stochastic differential equatior s (3,22). 


By analogy with (3.8) we obtain the following system cf differential equations 
for second-order moments: 


Ž, > Anat mÝ (ti t3 = 


(3.24) 
m m 
a 2,2, BY) BO mi (61, 5) (ku y= 1, 2, ..., n), (3.24) 


where, for brevity, the notation m(t) = m(t) is introduced. 


(x) 
The moments mj ja appearing on the right-hand side >f equations (3.24) 
become the autocorrelation ratios at the corresponding ir puts if ji = iz = j. 
Yj Jo these moments are cross-correlations, i.e., they characterize the statistical 


dependence between the forces at different inputs. An analc gous consideration applies 


to the moments mi? appearing on the left-hand side of the same equations. 
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§ 13. The Method of Impulse Characteristics 


In a number of cases the statement of the problem of random acting on 
a linear system is preceded by an analysis of its respons to a specific determinate 
input. Standard inputs are the unit step function defined by the relationships 
H(t)= Ofort <0, H(t)=1fort>0, and the unit impulse h(t) o unitarea, this [delta] 
function being the time derivative of the unit step function. The response of the 
system to a unit step function is called its transfer characteristic. Response 
to a unit impulse still lacks a generally accepted term. We shall henceforth call 
this response the impulse characteristic of the system. A simple relation exists 
between the transfer and impulse characteristics of the sane system, the second 
being We time derivative of the first. 


We shall examine first a linear system with one input and one output. We 
shall assume that the first n moments of the random applied input and the system's 
impulse characteristics $ (t)are given. The first n moments of the response at the 
output of the system are to be calculated. 


Let the input x(t) be applied at the initial time t = 0. We shall assume, 
for simplicity, that the energy of the system is zero at this moment. If 
this condition is not fulfilled, a damped regular process which can be calculated 
by known methods is superposed upon the investigated process. 
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We wish to find the response to the random input at the momentt,. The 
applied input can be considered as a succession of contiguous elementary 
impulses of infinitely short duration (Figure 1). Their properties are similar to 


aft) 


4 


Figure 1. The applied input as a succession of contiguous 
impulses 


those of the unit impulse. The system's response at the moment t; to an infinites- 
imal impulse which is at the distance t from the origin can be expressed as follows: 
dy (DÐ =$, — t) x (t) dt. (3.25) 
By virtue of the linearity of the system, the resultant response of the system 
to all the infinitesimal impulses y(t,), contaired in the interval (0, tı) is the super- 
position of the corresponding elementary responses of the form (3.25). Thus: 
4 


y= f OEM — N dt, (3.26) 


Expression (3.26) is an integral transformation of the random function 
x(t). If the first n moments of the input x(t) are known, then the general equation 
(2.52) of § 8 makes it possible to find the first n moments of the response y(t). In 
particular, we have on the basis of (2.45) for the first moment yt): 


3 t 
mP U= f MPO — ddt. (3.27) 
0 


The second moment [variance] of response yt) is determined from for- 
mula (2.50) as follows 


t h 


mP y= f f aP DE — Di — N) dti dh. ERAR 
o o 


For finding the general form of the moments of the system's response we shall 
proceed as follows. By analogy with (3.26), the response of the system at a time 
instant to can be written in the form: 

t 
yt) = f xO — Ade. (3.29) 


o 
The systems response at any other time instants tz t ty can be ex- 


pressed just as easily. Considering all these expressions as a set of n integral 
transformations of the random function x(t) which are similar to (2.53), it is 
not difficult to calculate, using expression (2.60), then-thorder moment of the 
system's response. In particular, we have for the second-order moment: 


ot 
mP (h, =f f MPE OG — OG — Md dts 
u 


v 


(3.30) 
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Thus, being giventhe moments of a random input and the system's impulse 
characteristic, and employing the properties of integral transformations, we can 
calculate the corresponding moments of the system's response to the mentioned 
random input. 


The above-listed relationships for moments make it possible to carry out 
calculations in the most general case when the response is nonstationary and 
caused by a nonstationary external force. When only the values of the moments 
of response at an infinitely long time after the input is switched on are of interest, 
the lower limits of all the integrals of this section should be taken as ~0O. Then 
the precise values of the upper limits become irrelevant. Only their correct order- 
ing in time is important. 


According to the remark make in § 8, the response of a system to a normally 
distributed applied input is also normally distributed. In the general case, an 
alteration of the distribution law takes place in a linear sys:em, 


The problem of random action upon a system with one input and several out- 
puts is solved in an equally simple manner. The responses of the system Yi Jav 
Ya at its n outputs at time instants ty ; toe sees rth are written by analogy with 
(3.26) and (3.29) as follows: 


t, 
»t)= Í xD — h at, 


vt) = f Ouh — h dt, 
° (3.31) 


e e e ‘Ñ 


la 


Yall = | Onla — bdt, 
v 
where EW, Ealt), RAA t L(t) are impulse characteristi:s of the system with 
respect to the corresponding outputs. 


The moments of each response are determined by re ationships of type (3.27) 
and (3.30). The mixed moment for the system's n responses can be found if we 
consider the expression (3.31) as a set of integral transformations of (2.53). 
Then the application of forthula (2.60) gives the ae result: 


marten Meng tof ie fine bercita X 
(3.32) 


X ti (f — t) e (fa — 4) -o En (fn — fnd dti dte... din. 
Let us now consider a system with two inputs and two outputs. If convenient, 


the results of its analysis can be generalized in a quite obvi>òus way for the case 
of m inputs and n outputs. 


We now introduce the impulse characteristic k(t) o° the system, where 
the first subscript indicates the output and the second subs::ript the input concerned. 
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In our case the system is characterized by the four impulse characteristics 
14), kolt). tglt) and Eaglt). 


We shall call the system's response to random excitation applied at one of its 
inputs its partial response Yi Here, as above, the first and second subscripts refer to 


the output and the input, respectively. The four partial responses of a system with 
two inputs and outputs are written as follows: 


Jue) = J x (H bu — t) dt, 


o 
t, 
Ya D= f aO — Nat, 
v 


fe (3.33) 
Ya D= f Otn l — hdt, 
v 


h 
Yar t= f xO bale — 1) dt. 
o 


The linearity of the system makes it possible to apply the principle of super- 
position and to express the total responses of the system at both outputs thus: 


Yih) = Yn 6) + Ya lti), (3.34) 
Va (ta) = Yor (ta) + Yn (fa). (3.35) 
Let us examine the total response of the system at the first output. The first 


moment of response y(t) equals the sum of first moments of the partial responses 
Yuti) and yalt). The latter are determined by equations of type (3.27). Hence, 


t, 6, 
mP U= f mP Oth —odtt fm” © ta anae @-99) 
0 u 


For finding the second moment we shall examine the response yy at both the 


instants to and ty 


(t) = t ty), 
Vi (te) = Yn C) + Yalta) (ssh) 
where the partial responses yilt) and Yifta? are determined by the first two 


equations of (3.33) in which t} is replaced by to. Taking into account equations 
(3.34) and (3.37), the required second moment can be written as follows: 


mY (t,, t) =M Ly Dy Cd) = M yn C) Ya CA + (3. 38) 
HM Y (ts) Yn CAH M Dy C) Yaz Gd Mie td Yia tN. 


By generalizing the result (2.60) to the case where each integral transformation 
(2.53) refers to a different function, we can express (3.38) in the following final 
form: 
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& & 
MP, = f f MPE, Miu, D ta GG — Mati dnt 
o WY 
te A 


+ ff me PE ta ha 0 dt dta 


o o0 
tt 
+ ff mi PO, Diu a i ba ai d+ 
0 o 


tt 
tf S AEE Da — ale — ha) dti dei, (3.39) 
u v 


(x4, x2) 
ts 
( 1 


where my t,) is the mixed moment of the applied inputs x, (t) and x(t). 


The calculation of moments of response at the second output is carried out in 
the same way. Analogously, in view of equations (3.34) and (3.35), we have for 
the mixed moment of responses yi (t) and Yalta): 


mor ty, t) = M19 (6) Ya D) = MV CD Ya GDH 


3.40 
+ Mlyn) Yo CAFM n CD Yee td EM se C49 
or in an expanded form: 
tot, 
ments, ta f f mE Obit Ee (a f) dti dt 
oo 
hb h 
+S S EEE, Otti — A a f — ta) dii d+ 
EREN 
+S S MEE Ota f — 0 te (a — Bah dh + 
ar 
+f S mE, Oath — A) beats — Aa dii dt (3. 41) 
Qo o 


The readily carried out generalization of expression: (3.39) and (3.41) for 
the case of m inputs and n outputs gives the solution of the most general problem 
of random input applied to a linear system. 


If stationary random processes at the output of the s:‘stem are analyzed, the 


integration limits in (3.32), (3.36), (3.39), and (3.41) shold be changed in accord- 
ance with the remarks following formula (3.30). 


$ 14. The Spectral Method 
Unlike the preceding sections, where the obtained results applied equally to 
stationary and nonstationary random processes, the randon functions considered 
in this section are stationary. 
Let the stationary function f(t) be definedin the interval (0, T) of the time axis. 


For simplicity, its first moment is assumed equal to zero This random function 
can be represented by a Fourier series in the stated interval: 


S= Dd (Arcos yt + By sin wy t), (3.42) 
k=1 
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where 


w= =%, (3.43) 
r (3.44) 
Ak =; JS (x) cos wx dx, ` 
a 
(3,45) 


B, = 4 f S(x)sin wx dx. 
v 


The amplitudes AL and B, of the spectral components, obtained by the integral 


k 
transformations (3.44) and (3.45) of the random function f(x) are random quantit- 
ties depending on the specific realization of f(x). 


Using equation (2.50) let us find the dispersion BA of the amplitude A, : 
k 


T 


2 4 
oA, = Af cos ON 2 dx, f my(x,—x,)c0s wX, dx,- (3.46) 
) 0 


We shall transform the double integral appearing in (3.46) as follows. Let 
us carry out the change of variables z = X37 Xj? in the inner integral. 
Then T ve 
ca, = =4 J COS wX, dx, fJ M, (Z) COS g (X2 — 2) dz = 


u T-T 


=p fan Tansee] ata Taak 


2,-T T,-T 
X cos w2 dz + f sin 2u,x, dx, f mo (z) sin wz dz | (3.47) (3. 47) 
a4-T 


Each of the double integrals of (3.47) has to be transformed by changing the 
order of integration. Considering that the function m,(z) ig even we have for the 
first integral: 


f ge g mlz) cos mz dz = 


D 24T 
= f M, (2) COS waz anf antf m @osoz dz f an= 
-T 
T 
=? f (T -— z) m, (Z) cos wz dz. (3. 48) 
Ge 


The two remaining double integrals are transformed in an analogous way. We 
obtain the result: 


af 
h= 2 zl (1 —F) me (2) cos 42 dz— 
ô 


T 
lı 2 
pa aa a S masin osz dz }. (3. 49) 
o 
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Taking a sufficiently long observation time T, one can make the term z/T within 
the brackets of the integrand of the first integral arbitrarily small. Similarly, for 
all the spectrum components of frequencies u,>a, where 2 is an arbit- 
rarily small positive number, one can reduce at will the role of the second 
integral of (3.49) by increasing the observation time, since thereby the order 
k of the spectrum component increases. Thus, for suffi:iently large T we have: 


on, = 2a,, 
(3.50) 
where a, is the amplitude of the corresponding componente of the cosine spectrum 
of the moment m,(z), i.e., 


T 
rf 
=> | m,(2)cosu,2 az. 
ük T J 242) Uk (3.51) 
Analogously one can find the dispersion of the amplitide Bk determined by the 
integral transformation (3.45). If the observation time T is sufficiently long, the 
derivations give results coinciding with (3.50): 
2 — 
Op, = 24g. (3.52) 
The spectral component of frequency ok of a rancom function can be 
written as follows: 


A, Cos wt + B, sin wt = Cy sin (wt + 9). (3.53) 
where 
a! 2, = Ak 
Ci = Ak + Bi; oy = arctg B (3.54) 


Since the average of the sum equals always the sum of the average values of 
summands, we have on the basis of (3.50) and (3.52); 


2 
See = 04, + of, = 4p. (3.55) 


Relationships (3.50), (3.52) and (3.55) make it possible to make the 
following statement whichisimportantfor the spectral theory of random processes: 
the spectrum of the mean squares of the amplitudes of a Fourier series, or, in 
other words, the power spectrum of a stationary random function, coincides 
(to within a constant factor) with the cosine spectrum of its second moment 
or correlation function. 


Consequently, by analogy with the spectral theory of ceterminate functions, the 
conclusion can be drawn that a weakly correlated random “unction has a broad 
spectrum while a strongly correlated function has a narrow spectrum. This state- 
ment will be given a clear physical interpretation in the following. 


Let us continue the investigation of the spectrum of :ı random function. 
We shall first clarify the question of statistical dependence between the probability 
amplitudes A, and Bẹ. To this end we regard expressions (3.44) and (3.45) 


as a set of two integral transformations of the random unction f(t) and employ 
relationship (2.61) to calculate their second-order mixec moment: 


TT 
M(A,8,) = rs f My (Xz — Ly) COS Wy X, SIN Wp AX, Ay. (3. 56) 
o o 
Expression (3.56) can be transformed like (3.46). The result is: 


T 
(3.57) 
M{A,B,) = — + f F m, (2) sin wz dz. 
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In view of equations (3.50), (3.51), (3.52) and (3.57) we note that for 
T — œ the correlation coefficient 


M(AgBa) 
ptAg, By) = “l2 (3.58) 


OA go By 


tends to zero, i.e., for a sufficiently long observation time the random amplitudes 
Ak and By can be considered as statistically independent. 

The question ofthe statistical dependence between the amplitudes of spectral 
components of different order, e.g., A, and A m? Bk and Bn’ Ak and Bm and, 


finally, An and By can be investigated in an analogous way. For T — œo allthese 


amplitudes are statistically independent of each other. Thus, under this condition 
a random function is resolved into statistically independent components. 


We shall turn now to quantitative relationships in the spectrum of a stationary 
random function. According to (3.55), the mean square of the instantaneous value 
of the k-th spectral component can be written as: 


a= F oby = 2p. (3.59) 


Since the moment m,(z) is usually a damped function, the values of ak and 


consequently of 6 2 are very small for a long observation time T. This con- 
clusion follows directly from expression (3.51). However, for large T the spectral 
lines lie very close together, and their number in a not too small frequency 
band is quite considerable. 


We shall select from the spectrum of the random function a narrow frequency 
interval SF within which the spectrum can be considered as uniform. The number of 
spectrum components in this interval will be AFT. Therefore, in view of (3.51) 
and (3.59) we can express the mean square of the random function in the frequency 
interval AF in the following way: 

T 


ofp = of AFT = 2AF Ta, = 4AF f m, (z) Cos wz dz, ia; 


where ok is the frequency of one of the spectrum components in the interval AF. 

We shall introduce the spectral density F{m) of the random function, inter- 
preting it as the mean square of the random function in a unit interval of the an- 
gular frequency. From expression (3.60) we have: 


2 T 
OP 2 

F (wx) = -JREF = zf m, (2) COS wz dz (3.61) 

v 
or, going over to the limit for T —> 00, we obtain: 
X% 
2f ; d 
=- mMm,(z)coswz az. 
Fw) =F 2(2) (3.62) 


Thus, the spectral density of a stationary random function is a Fourier 
transform of its second moment. If the spectral density F(w) of a stationary 
random function is known, one can find its second moment by using the in- 
verse transformation: 


m,(z)= f F (w) cos wz dw. (3.63) 


The mean square of the random function is expressed by its spectral density thus: 
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o? = F (wu) du. (3.64) 
J 


The results obtained, and in the first place the relat onships (3.62), (3,63) 
and (3.64), make it possible to develop a spectral theory of stationary random 
processes, 


Let us examine a system with one input and one outpit subjected to a station- 
ary external force. It is required to find the steady-state response of the system. 
As in the spectral theory of dynamic processes it is ass:imed here that the complex 
transfer ratio of the system is given: 

Y Ge) 
Kijo) = Xa" (3.65) 
where X(jw) is the complex amplitude of the harmonic applied input, Y(jw) is the 
complex amplitude of the steady-state harmonic response əf the system. 


Knowing the spectral density Fin (w) of the applied input or having calculated 


it with the aid of expression (3.62), and being given the second moment of the response 
one can calculate the spectral density Four! ™) of the syst2m's response 


Fou (w) = Fin (w)| K (Joy PÈ. (3.66) 


If the spectral density of the system's response is kiown, expression (3.63) 
makes it posgible to calculate its second moment: 


oa 
miz) = f Fou (w) cos zw dw (3.67) 
v 
and, in particular, its mean square 
o= mP (0) = f Fou (o) do, (3.68) 
a 


The above considerations have been elaborated independently of the results 
of the preceding section obtained in working out the me hod of impulse character- 
istics. Another approach is possible in which one starts from the relationships 
of the preceding section and bases the spectral method upon them. We shall 
examine this approach dealing, as yet, only with systems which have one input and 
one output. 


In the preceding section we obtained formula (3.3C) for the second moment 
at the output of the system. Taking into account the stationary nature of the processes 
dealt with we shall rewrite this formula, taking -- o9 ai! the lower limit 
h fi 


mP, t= f fara. E — AE (lg — 2) dti dta. (3.69) 


We can obtain from expression (3.69) the above-shown basic relationships of 
the spectral method. Before proving this we shall apply :urther transformations to 
(3.69) using the property of stationarity of the process :onsidered. 


In stationary processes the second moments appearing in (3.69) are even 


functions of the differences tot and. t37 ti Further, since only the relative 
position of the time instants t, and ta is important, one can assume t, = 0 


without loss of generality. Now, introducing the notation : = to - ty = to and chang- 


ing the integration variables: t} = -0ta =-6,, we can write (3.69) in the form: 
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mP (2) = f 6, +2)00, m (0, — 0) E (0,) 20, (3.70) 


or, if we set 6, — 0, = 9, 


m” (z)= f tataan fin (0) E (8 +-0,) d0. (3.71) 


-6 
Let us change the order of integration in the double integral of (3.71). Then 
we shall obtain; 


m” (z) = f enO F EO HODEO, +240, + 


= (3.72) 
fare J FO + ODEO + 2) 8, 


In the inner integral of the first term (3.72) the integrand vanishes for t<- 8. 
Therefore the lower limit of this integral can be formally replaced by —oo. 
A similar change is equally possible in the inner integral of the second term since 
for §,<—z wehave (0, +2)= 0. Consequently, 


+œ 
m” (z) = feon J FO + 6), + 2)d0, + (3.73) 
e. 
+ i mi (040 f (0 + 8,)(0, + 2)d0, = 


+00 
= fmPon J ECH ODEO + 2)d6,. 


carrying out the substitution ð+ ft =y, in the inner integral of (3.73) we 


obtain: +00 +00 
mY (2) = f Poa J eneit d= 
Pi a +0 
= f mP Ove, 2) 40, (3. 74) 
where +o 
0. ae EHE — O + 2) dy. — 


We have now completed the preliminary transformation of expression 
(3.69) and proceed to the basic exposition of the spectral method. 


As well-known /11/ the complex transfer ratio of the linear system and its 
impulse characteristic are connected by the direct and inverse Fourier transforms: 


+œ 
K(jo) = f t(tye 2" dt, (3.76) 
pe 
=y J Kme” dw. (3.77) 


Furthermore, the following relation 1s known in the theory of Fourier transforms: 
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+œ +œ 
f sO+oday f Yale” de, (3.78) 


where fis the spectral density of the function f(t): 
+0 
fa= | IO at. ponte 
-0 
Making use of these results we can rewrite equation (3.75) thus: 
+a 
20 =a f IKUO e * de. (3.80) 
-œ 


Substituting this expression for ẹọ (f, z) in (3.74) and changing the order of 
integration we obtain: 


+0 +œ 
m” (z)= f m$ (8) do. ot J IK (Jo) | etme -10 dw = ee 


+7% +00 
=F f lke) pda. E fJ ema. 


Since the second moment m, %0) of the applied inpu: is even, then, taking 


into account (3.62), the inner integral of expression (3.81) can be transformed in 


the following way: +œ 2 œ 
1 = = 
= f m=) (0) e $= d0 = — f m O) cos 8 dô. (3. 82) 
-0 v 


The integral of (3.82) is an even function of the frequency w. Similarly, the 
square of the modulus of the system's transfer ratio |K (jm) |° is an even function of 
the frequency. This makes it possible to write expression (:..81) in the following 
final form: 


a œ 
; 2 

apasi, | K (jw) |? cos wz dw » — J mt (0) cos m8 d9. ERT 

This result is entirely equivalent to the one obtained errlier for the relation- 
ships of the spectral method. In fact, comparing integral (: .82) with expression 
(3.62) we notice that the integral gives the spectral density of the applied input. 
Further, taking into account expression (3.66) we see that (3.83) has the same 
meaning as the earlier result (3.67). 


The difference between the second way of establishing he spectral method and 
the first one consists in the fact that in the first one the concept of the spectrum 
of the random function itself was introduced while in the second one appears the 
spectrum of only its correlation ratio. Therefore, the secon-ione leads us more 
quickly to the goal, However, in some cases the spectruin of the random func- 
tion is a very useful concept. 


Let us proceed now to linear systems with several inpits and outputs. For 
the sake of simplicity we shall examine here, as hitherto, systems with two inputs 
and two outputs. The generalization of the spectral method which we have 
in mind can be achieved by any one of the two ways given above. Here we give pre- 
ference to the second one. This is not due to basic consider: tions but is explained 
by the desire to shorten the calculations necessary for obtairing the final results. 


Our system will be characterized by four transfer ratios: 
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[K (jo) |, Ik, Gol, IK, Go| and IK, zw]. and also by the four phase shifts: 


914), Mylo), Poy(w) and Po9(m), where the first and second subscripts refer to 
the output and the input, respectively. 


In the preceding section we obtained expression (3.39) for the second mo- 
ment at the first output. Taking into account the stationary nature of the processes 
considered, this expression can be written as follows: 

tow 
a , 4 (a , r 
mP, t= f SMP E Din an (fat dti dh + 
=W -%0 


noh 


+ f SAEPE Dafa i — dt d+ 


par 
+ f SEE Ota Ca — i ta a — f dti dts + 
“ean 
+f SAE Dal — t) tia (fa — ti) dt dti. (3. 84) 


The first and the fourth integrals of (3.84) have the same form as the integral 
of (3.69). Therefore, by analogy with (3.83) one can write: 


h t 
, (3.85) 
f f ms" (thy 12) Eni (a — Ci) En Cfa — 2) dt dtg = 
TA w œw 
= f 1K, (io) coswzaz: 2 f mž’(0ycos wð a8, 
a Se : 
f [AEE Dat -Odas 
-0 -œ 
œ œ 
= J |K,2 (ju) [cos wz dw» 2 f mi (0) cos w dð. (3386) 
vu u 


The integral of (3.85) gives the second moment of the response that would have 
taken place ifexternalforce were applied at the first input only. Integral (3.86) 
is aresponse analogous to the characteristic for external force applied at the 
second input only. 


We shall proceed to the transformation of the second integral of (3.84). Unlike 


equation (3.69), where the function m,” t ty) was an even function of the 


(xy, X2) 
2 


difference ty = te here the function m (th - t'o) although being a function of 


the difference ty-ty (by virtue of the stationary nature of the process), is not in 


general an even function. Since in the course of the transformations which led us 
from equation (3.69) to (3.74) and (3.75) we had not assumed that the function is 
even, we need not repeat the above transformation and we can write directly: 


fr, t 
J [AE PG. Dt O a — i) dhi d= 
+0 


= f ae = (8) (0, z)d8, (3, 87) 
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where 


+0 
90, i=] tia (H) n ($ — 0 + 2) dẹ. (3.88) 


For the further transformation of expressions (3.87) and (3.88) we shall use 
the relationship +0 +00 


f sMet+oyde= ze f Seca, (3.89) 


=- 


which is known from the theory of Fourier integrals, and in which f, and g, are 
the spectral densities of the functions f(t) and g(t) and in which the asterisk 
indicates the complex conjugate. 


Since, according to expression (3.76), the system's transfer ratio is given by 


the Fourier transform of its impulse characteristic and in view of equation 
(3.89), we can rewrite equation (3.88) as follows: 
+œ 


? (5, 2) = x J Ki2(jo) Ku (Jw) etm o- tw, (3. 90) 


-% 
We now substitute the obtained value of ẹ(0,z) in (#.87) and, changing the 
order of integration,we have: 


S [ERÉ uan A at d= (3.91) 


-œ -~ 


+œ +0 
S Kaien nnd dey Jf mer (0) e74 ta. 
An analogous form can be given to the third integral cf expression (3. 84): 


f J mi = 0, £3) ban (i — 11) Era (tg — t3) dti di3 = (3. 92) 


=7 f Kir Jo) Kiz (Joye ™ dw- L mr a) eI" d0. 
Now, using the results of (3.85), (3. 86), (3.91) and (3.92) we rewrite 
(3.84) as follows: 


mY (z) = Î [Ky (Jw) |? cos wz dw» 2 f mS” (8)-08 wð d8 + 
+f ike tes wzdo. = 2 f aoo w0 dð + 
i} rs Ad 2 
+4 f IKu o) Kis (Jo) + Kis (J) + Ky aiet dw X 


XL f mgema Oenea. (3, 93) 
-œw 
‘et us analyze the equation obtained. Comparing tl.e first integral of (3.93) 
with equation (3.83) we conclude that this integral gives he second moment of 
the response which would take place if the external force w re applied at the first 
input only. The internal integral of the first term multiplied by 2/r gives, as 
follows from (3.62), the spectral density of external force at the first input: 


2 2f m= (0) cos wh dd = Fa, (w). (3:24) 
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The second integral expresses the second moment of the response under 
external force at the second input only. Similarly to (3.94) we have: 


2 f m$” (6) cos wb dð = Fz, (w). (3.95) 


o 
The third integral is the result of the statistical dependence between the ex- 


ternalforceatthetwo inputs. The inner integral of the third term witha factor of 1/n 
+0 


= f mE Ed (N) Je dh = Fer, (jw) (3. 96) 


expresses the spectral density of the mixed moment m1: x2) (8). We shall call 


this spectral density the mutual spectral density of the external forces. Since in 
general the mixed moment is not an even function, transformation (3.82) is not 
applicable here and the mutual spectral density itself is complex. 


Taking the aforesaid into account, we can write equation (3.93) in the 
following more compact form: 


m% (2) = J | Kii Jo) |? Fr, (m cos wz dw + (3.97) 
u 


œ 
+ f | Kie Jom) l? Fz, () cos wz dz + 
o 


+o 
+y J Kao) Kae Katja Ka GaN Faa, (jo) Pd. 
-œ 


An analogous relationship is valid for the second moment of response at the 
second output: 


my (2) = f IKa a)l Fal (ocos wz de + 


o 


+ fi Ky, (Jo) |? Fz, (w) cos wz dw + 


+00 
+E f Ka (ia) Rie ju) + Kin Ge) Ka Uo) Fry (de) Pde, (3.98) 


To calculate the mixed moment of response at both outputs of the system we 
use equation (3.41) of the preceding section which assumes the form 


mi ty) = 
toot 


= f fa. MiG D En (a — 0) dti dt + 


-w -0 
ho oo& 
tf S aE total — in le — fa) dti dia + 
a <p ; 
+ f f mS = (0, £2) San i — C) Eze (f2 — t2) dt dtz + 
-%0 -0 
no R 
+ f SEE talh — 6 e (fa — A) di dth. (3. 99) 
-0 -w0 


for stationary processes. 


The transformation of analogous integrals had been carried out in the 
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foregoing. Therefore, without writing out the corresponding derivations, we shall 
give immediately their final result: 


+œ 
m (= f [Ki (io) Ka Ce) Fe, (0) + 


+ Kia Jo) Ku (J Fg, () + 


+ [Ki (Jw) Ky) (jw) + Ki: (jw) Ka (jo)} Fez (j) e ido. 
(3.100) 
It is understood here that the order of the two time instants is such that to > tr 
It is easy to see that the square brackets in the integrand of (3.100) contain 
the mutual spectral density of response at both outputs. 


Thus, Equations (3.97), (3.98) and (3.100) obviously determine the statistical 
properties of the responses at both outputs of the system, 


§ 15. An RC Circuit excited by a Stationary Fluctuating Voltage 


In this section we are concerned with application of methods expounded in the 
preceding sections to a simple particular problem. This problem is given here as 
a clear physical ilulstration, being at the same time the point of departure for some 
generalizations to be given later. 


u,,(t) ¢ Uour(t) 


Figure 2. An RC Circuit excited by a fluctuating voltage 


Let an electric fluctuating voltage u;,,(t) be applied at the input across the RC 
circuit (Figure 2) at the moment t = 0. It is assumed that this voltage is a station- 
ary random function of time, whereby its d.c. component (first-order moment) 
equals zero, and the second-order moment, which in this c ise coincides with the 
correlation ratio is 


m\”) i. td) = 0, ee! hens f (3.101) 


It is easy to see that the quantity oj, is the square oi the effective value of 


the input voltage. The initial charge on the capacitor is assumed equal to zero. 


It is necessary to find the steady-state law of the second moment of output 
voltage. This will make it possible, in particular, to find the law of the variation 
of the effective fluctuating voltage at the output. 


We shall use the method of stochastic differential equations for the solution 
of this problem. The stochastic differential equation which describes the investigated 
random process has the form 


du ou (f) 


At Bow (f) = aiin (2), (3.102) 


where a= 1/RC 


Thus, the operators A, and B, of equation (3.1) are in this case expressed by 
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Aa a +0; Baa. (3.103) 
Therefore, the differential equation (3.8) reads in this particular case 
($+ a)(¥ +a) m mE (6, ty) = amf (ty, ty) (3.104) 
ðt, Oty t °2. LORE Zd 


To represent the required second moment in the form of (3.21) it is necessary 
to find the double transform of the second moment of the input, determined 
by expression (3.101). Let us calculate its Laplace transform to the 


variable t: 


œ 
mo”) Pa t= f mi) (ty, la) eat, = 
0 


h æ 
= 0 f ete- e-Ptat, + Jete e-pt a} 
0 





o$ 
rag (Bemat — GB pe], (3. 105) 
t 


Now we shall transform the obtained expression with respect to the vari- 
able t,: 


ms” (Py. Pa) = fr ms” (Py, fa) erh dh = 


130 
=y 2p feer ats + po f etme Wit.) = 
1 v 


EERS 


=on VESIES Dy, +P" (3. 106) 


The transition from the operators A, and B, to their transforms gives, 
according to (3.13) and (3.15): 


=p+a; B, = 2. (3.107) 
Therefore, the required double transform of the second moment which has in 


general the form of (3.21) is obtained in this case as 


23 + Pit Pa 
(Pr + a) (1 + 5) (Pa + Po) (Pz + 2) (Pe + BY" 


The inverse double transition to the original, which can easily be carried out 
with the aid of a table of operational relations, leads to the required result: 


(Pie P3) = 273), (3, 108) 





2 
mE Ot), ty) = sp | ae-P1 41 petiti 
ae 
+ (a+ Bye-ethet —afe(Pltetd + e-leh + Bla) (3. 109) 


Let us examine the obtained expression. As should have been expected, it 


is symmetric with respect to the variables ti and ty and satisfies the initial 


condition m (out) (0,0) = 0. The second moment of the response depends, not only upon 
the absolute value of the difference It, ~t il but also upon the disposition of both 


time instants with respect to the time k input is switched on. Thus, there 
is a nonstationary random process at the output of the circuit. The steady-state 
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law of the mean square of the output voltage is obtained by setting in (3.109) 
t, = t, =t: 
1 2 


p a -ot 
Foul) = IF? sîn {! = Re (ae-Bt — Be “ah. (3.110) 


2 
remains finite, the second moment of the output voltage ‘ends to its steady value: 


2 
(out) 


ag. 
MPR as ta) = age lae Hb — pere ty (3.111) 


If tae and t, -»ccwhile the absolute value of their difference ioe tl 


Now the second moment ofthe responsedepends on y upon the relative position 


of the time instants t and ta» i.e., in the limit a staticnary random process is 


obtained. 


Assuming in (3.110) t —= œ or in (3.111) It. - tl = 0, we find the mean 


square of the fluctuation steady-state voltage at the output of the circuit: 


2 
Sour = Bin ats’ (3.112) 
The normalized steady-state correlation ratio will be: 


moat) (tp ty) oe PTA get! fhi (3.113) 
Pout (ty, t) = ——— eo gg 
out 


Thus, the necessary quantitative relations have be :n obtained. Let us analyze 
them. 


We shall call correlation period the time interval within which there is a signi- 
ficant statistical dependence between the values of the random function (voltage inour 
case). As a criterion of this dependence one may take a(lt, - to ) > 0.1, for instance, 


Furthermore, let us call the time in which the voltage of the capacitor is reduced to 
10% of its initial value, the discharge period of the cap«citor. Then equating the 
correlation period of the input voltage to the discharge period of the capacitor 
corresponds to equating 2 and $. 


Let B<<a, i.e., the correlation period of the app ied input greatly exceeds 
the discharge period of the capacitor. In other words, tie speed at which the random 
process occurs in the circuit is much greater than the mean rate of change of the 
input voltage. In this case we obtain from (3.112) and (2.113): 


cpa =a: Pout (t,, la) x e768! h-t, LE (3. 114) 


Equation (3. 114) signifies that under the indicated conditions the statistical 
properties of the output voltage coincide with these of th: input voltage. The reason 
for this lies in the fact for 6 <2 the voltage at the cutput of the circuit 
manages to follow the variations in the input. 


We shall assume now that a << 3, whichcorrespor ds toa long discharge period 
of the capcitor as compared with the correlation period of the input voltage. 
Then equation (3.113) reduces to 


Pout (hy. ty) = ew tl enh l (3.115) 
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i.e., the degree of statisticaldependenceatthe output does not depend upon such 
dependece at the input, but is determined exclusively by the parameters of the 
circuit. 


It is noteworthy that with weak correlation of the applied input, (7< 6), 
the equation of the correlation ratio of the output voltage has the same form as 
the equation of the discharge of a capacitor through a resistance {.e., the equation 
of the specific transient process of the system. This correspondence is not accidental. 
The reason for it will be explained in the next section. 


Let us note that when speaking of a weak or strong correlation of the 
input we should not consider its correlation period unrelated, but we ought to 
compare the correlation period with the duration of the specific transient pro- 
cess of the system to which the excitation is applied. 


my” | tg - ty| 


4 t, 


Figure 3. Graph of the second moment of the voltage across the input of an 
RC circuit 


Let us explain yet another characteristic property of a weakly correlated in- 
put. For this we shall turn to expression (3.112) which, for a <5 can be written 
thus: 2 


of =a Oin 
our = jà 
r (3.116) 
For a clear interpretation of the relation obtained let us compute the area S 
bounded by the graph of the input moment (3.101) and the abscissa (Figure 3). 
We shall put there |tọ-tı|= 1. Then 
2 


S=2f Herero. (3.117) 


taking the obtained result into account we shall rewrite expression (3.116) in the 
following way: 


a 1 
ou = yas. (3.118) 


Thus, the mean square of the system's responseto a weakly correlated 
input is determined by the area S, which characterizes thisinput and by the para- 
meters of the system. The present result was obtained for the specific system 
with the particular form (3.101) of the function m,m (tisto). It is shown in the 


following section that this statement is of general application. 

All the results listed above have been obtained by the method of stochastic 
differential equations. Let us solve the same problem by the method of impulse 
characteristics. To make derivations brief, we shall compute the second moment 


of theresponse onlyin the stationary regime. For this we shall use expression 
(3.30) in which the lower integration limits are taken equal to — cv. 


We shall assume for the sake of definiteness t > ta and we shall calculate 
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the inner integral of expression (3.30) with the above-ment.oned change of the 
lower limit. The impulse characteristic of an RC circuit s, as well known, given 


by: 


b(t) = een, (3.119) 


Taking into account relation (3.101) for the second momentof the applied input 
we have: 


t, 
J nPE Didi = 
-09 ` 
lio ehani ; 
= asi, Í etU ete a 
-œ 
t . , . 
fre eteo gl Vee 


13 





saat, oF eeh (3.120) 


According to (3.30),the result obtained should be multiplied by ae C1 —'2) 


and integrated with respect to to between the limits — oo and to: Carrying out the 
said calculations we finally obtain: 
ee 
mae te fa) = aya laa Pf) — Beet =t, (3.121) 
i.e., an expression which coincides with (3.111). 


Let us carry out the same calculations by the spect"al method. First, using 
(3.62) we shall find the spectral density of the appliea inp it 
O 


2 2 2 3 
Fin (œ) = 2 f de-Meoswz dz = 2 o, at ate (3.122) 
0 
The transfer ratio of the circuit equals: 


1 a 
KUDI = FTF aRO T Var (3.123) 


Therefore the spectral density of its response is, according to (3.66): 


2 
Fou (0) = = sia BET T TF oat) (3.124) 


Now, using expression (3.63) we find the second mo nent of the steady-state 
response of the circuit: 


(out) Ne 2 3 278 cos zw pare 
ve w= fis GT oH Pay A= 
aa? 
= rage (e pe). (3.125) 


This result had been obtained earlier by the method of stochastic differential 
equations (formula 3.111) and by the method of impulse caracteristics (formula 
3.121). 
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$ 16. Uncorrelated Input 


In many problems of the theory of random processes the input is weakly 
correlated. In this case it is both possible and expedient to idealize the pro- 
perties of the input. This idealization leads to the concept of $-correlated or 
uncorrelated input. 


We shall use the method of impulse characteristics to carry out the 
mentioned idealization. We shall turn to the general expression (3.30) and subject 
it to a transformation for the case of weak correlation of the applied input. Let 
us examine the inner integral of this expression, where we set to > th 


Weak correlation of the input signifies that the major part of the area S, bounded by 
the graph of second moment mg (t'e t'a) and the abscissa, spans a narrow inter- 


val of the values of ti which contains the time moment tz- The narrowness of the 
mentioned interval should be understood in the sense, that the variation of the 


function £ ttp in this interval is insignificant, and it can be replaced in this 
interval by its value at the point t}, i.e., by the constant Elt- ty). This makes 


it possible to replace the inner integral by the following approximate expression: 


H : (3.126) 
J mPa. Ue — Ade SEG — a) f mG dt = 
o 0 
= K(f — h). 
where the area S is in the general case a function of t). The weaker the 


correlation of the input, the more exact is the estimate (3.126). Formula(3.30) 
now assumes the form 


t 
mI (tr) f SEDE — G a — dt, aua 
0 


The result obtained confirms the statement of the preceding section (follow- 
ing formula (3. 118)) that, when a weakly correlated random input acts upon a 
system, the area S fully characterizes the random process. 


Since the form of the boundary of S is irrelevant under the conditions consider- 
ed, the actual moment ma Nt to) may be replaced by an impulse function of 


equal area, i.e., 


mP (K, ta) = Sè (a — ti), (3.128) 
where 8 tytn is a unit impulse. 


An analogous situation exists in the theory of determinate transient processes. 
If the system is acted upon by an impulse the duration of which is much shorter 
than the duration of transient processes in the system, such an impulse may be 
replaced by an impulse function, i.e., an impulse equal in area to the actual one, 


having infinitely small width and infinite height. 


An input whose second moment is of the form of (3.128) is called ĝ—correl- 
ated or uncorrelated. The estimate (3.127) becomes exact for such an input: 


h 
mP = f SDE — tD El — f) dhh. (34129) 
o 


In the particular case when the applied input is stationary we have: 
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& 
mP =S f U4 DEl t) at. ee 
o 


If the steady-state second moment of the system':: response is sought, the 
lower limit of the integral of (3.130) should be replaced b;; —oo, i.e., 


h 
mP =S f Uh DE É h. (3.131) 


-w 
The last equation can be presented in a form which is more convenient for 
computations. We shall set tta = t and introduce a new integration variable 
x= t-t. Then we obtain: 


m” (s) = S f R(x) E(x -+ 3) dx. 
; (3.132) 


The last equation gives the connection between the steady-state second 
moment of response and the impulse characteristic of the system if the input 
is uncorrelated. 


Other relationships of the method of impulse characteristics are easily trans- 
formed in a way analogous to the above. 


Let us see how the general relations of the method of stochastic differential 
equations become simplified for uncorrelated input. Fer this kind of input its 
moment in differential equation (3.8) which connects the second moments of 
output and input should be replaced by the value of the moment from (3.128). 
In accordance with this, let us calculate the double transform of the second 
moment of the input, entering in expression (3.21). We shall assume here the 
input as stationary. The transformation with respect t» the variable ty gives: 


mi (py ta) = f Sèta — h) e?e dt, = Se- Ph, (3.133) 
v 


A second transformation with respect to the variable 2 leads to the following 
simple result 
mP (s = e: 
i Pit Ps (3.134) 


Let us note that this result could have been obtaine¢c algo from (3.106), in 
view of (3.117) and assuming § — 0. 


The above considerations can be obviously extended to uther relationships of 
the method of stochastic differential equations. 


Let us find the spectral density of uncorrelated input. If the correlation 
period of the input is much shorter than the period of the frequency at which 
the spectral density is to be computed, then one can aisume coswz= l, in 
the calculation of integral (3.62) and then: 


F(w) = 2 f my(z)dz = z 
a 


(3.135) 
where the quantity S has the same meaning as before. 


In the ideal case when the correlation period equale zero, expression (3.135) 
ig correct at all frequencies from zero to infinity. Fora small but finite correlation 
period it is valid only at sufficiently low frequencies which correspond to periods 
much longer than the correlation period. 


Equation (3.135) shows that a stationary uncorrelated input has a uniform 
spectrum in the entire frequency range from zero to infirity. 
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An uncorrelated input has an infinite mean square of. as follows directly 
from equation (3.128) when we take there t= t'}. The same result can be 
obtained from (3.117) by assuming B-»0co at S= const. Consequently, 
such an input has infinite energy. This result is the consequence of the assumed 
idealization of the actual properties of the input. 


mg (tz - ty) 


Aa 


at 
at 
Figure 4. Graph of the second moment of an uncorrelated voltage input 


We shall pay attention to the fact that an uncorrelated input can cause a 
finite response of the system only if it has an infinite mean square, i.e., afinite area 
S. To explain this we shall introduce the concept of specific energy of the input, 
interpreting it as the energy developed in a unit active resistance. If we now 
consider the input succession of infinitesimal impulses immediately following 
on each other, the heights of the impulses being considered as uncorrelated, then 
the graph of the second moment will be of the form shown in Figure 4. and the area 
S will represent the mean specific energy of the individual elementary impulses. 


Each elementary impulse communicates a certain energy store to the system. 
Storage of energy takes place in the system only when the impulses applied to it 
are ordered to some extent, as is the case with determinate as wellas with correlated 
random actions. Then, at infinitely small specific energy of the impulses (finite 
a2 ) their superposition produces a finite effect at the output. In the absence of 
correlation no energy can be stored in the system and finite response can be eli- 
cited only in the case when the specific energy of each elementary impulse is infinite, i.e., 
Fin co. 

Let us explain the connection between the form of the correlation ratio at the 
system's output, under uncorrelated input, and the character of the system's 
specific transient process. For this we shall turn to the RC circuit examined 
in the preceding section. Let there occur a stationary random process caused 
by an uncorrelated fluctuating voltage in the circuit whilst the voltage across the capacitor 
equals u, at the moment ty (Figure 5). The voltage u, across on the capacitor at the 


moment to can be regarded as the result of the superposition of two processes: 





Figure 5. The character of the voltage at the output of 
an RC circuit under fluctuating input 
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1) the discharge of the capacitor through the resistance anc input source during 
the time tort, (determinate process), and 2) the simultaneous charging of the capacitor 


by the input fluctuating voltage (random process). 
Thus: 


uz = u,e -®t 4 hande (3.136) 
Now, multiplying both sides of (3.136) by u and taking the average we obtain: 


M luu) = M| uiet 4 M fuian]. (3.137) 


The left-hand side of (3.137) represents the second n:oment of output voltage 
m, (0t) (ty, ty). We further have: 


M [ai] = obu (3.138) 
and 


M (utana) = 9. (3. 139) 


The latter relation results from the fact that at uncorrelated input there 
ig no correlation between the voltage Und resulting from -nput fluctuations over the 


period t and the voltage u, caused by these fluctuations at the moment t, . 


-t 
2 1 
Taking these considerations into account, one write equation (3.137) in the 
following form: 


me) (ty, tmou eE, (3.140) 


whence we have for the normalized correlation ratio of the output voltage 


mor) (ty, fa) 
2 


Pout (ty. la) = = ents) (te > t), (3 4 141) 


out 
i.e., an expression which coincides with (3.115). 


It follows from the given results that under an uncorrelated input, the 
correlation at the output of the system results from some measure of resi- 
dual response to the earlier applied input. This residua. response decreases 
according toa law which coincides with that of the system's s ecific transient process, 


To conclude this section we would like to remark hat in §8 a random 
function was mentioned, the arbitrarily near-in-time valves of which were 
statistically independent. Such a function is uncorrelated and all the considerations 
given above are applicable to it. However, if the second m>ment hag the form of 
(3.128) this is not yet sufficient for inferring that there is ro statistical dependence. 
Such an inference is valid only for a normally distributed fı nction. 


$ 17. The Problem of Two RC Circuits with a (ommon Input 


The aim of this and the following section is to give an illustration of the 
investigation of random processes in linear systems by me‘hods given in this chapter 
as applied to systems with several inputs and outputs. We shall now examine 
the following problem. Two RC circuits are given in (Figure 6) with the para- 
meters: 
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l 1 
a= RE ERG (3.142) 


A, 





uf} kon lt) 


boul U 


Figure 6. Two RC circuits with a common input 


A random voltage ut) of a correlation period much smaller than the time 


constant of the circuits is applied simultaneously at the input of both circuits. 
It ig required to find an expression for the mixed second moment of the response at 
both outputs of the system. As in the preceding section, we shall give a solution 
of the stated problem by using each of the indicated three methods. 


The behavior of the examined system is described by a system of two stochas- 
tic differential equations of the first order: 


du out t (t) 


dt + Qiu on 1 (f) = Ilin (t), (3. 143) 
du (t) 
— + Boll our a(t) = Qilin (ft). (3.144) 
Let the voltage Youth be taken at the time t and the voltage Uoute at to: 


We then have for the sought-for mixed moment the following differential equation 
ð at, i 
(3, ta) (3 tae) mi" Pi, td = aam (ty, ta). (3.145) 
at, Oty 


Applying formula (3.134), we obtain the double Laplace transform of the 
second moment of the response. 


ayaoS 


(out 1,2) a J 
ma (Pr Po) = F a pa + a F a (3.146) 


Returning to the original with respect to the variable P) we now obtain: 


me p) = oO (e-t — e-Bay, (3.147) 


Assuming ty > ty for the sake of definiteness, using the delay theorem of 


operational calculus, and then returning to the original with respect to the varia- 
ble Py we finally obtain: 


me" r Dt, t) = Te naa Oaa l (1.48) 


By virtue of the full symmetry of both outputs in the case of f, > ¢, we can 
write without further calculation: 
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mor De t= EL S fealty — ent tata], 


1+0, (3.149) 
The obtained results (3.148) and (3.149) are similar in some features to ex- 
pression (3.109) obtained in the preceding section. We im mediately notice they 


satisfy the initial condition m, (out 1, 2) (0, 0)=0. Further both expressions have 
two terms in square brackets, the first term being a funct:on only of the difference 
(ty = ta) and corresponding to a stationary random process which approaches the 


steady-state value for t, — 00 and tə — oo. The second term, on the other hand, 


1 
tends to zero in the same limit. 


The nature of the statistical dependence between the responses can be easily 
traced by writing out the expressions for the normalized correlation ratio of 
response. From(3.148) and (3.149) in view of (3.125) we have in the steady state, 
att, > ty 

a 2 ¥ 193 6 -a0t,- h) 
Pour a, 2 (fy, aa vars à s 


a (3.150) 
at toot, x Vix 
= SEES e-lh -h 
Pou i 2 fe) = F a emote f, (3.151) 
As should have been expected, both expressions yield the same result at 
testa 
1 2 
2y aa 
Pout max = Gag” (3.152) 


The strongest statistical dependence, which becomee functional in the limit. 
occurs when @, > %4 (p-»1). As can be seen from comparison of relations (3.150) 
and (3,151), the steady-state correlation ratio of both responses is not, in 
distinction from the autocorrelation ratios of each response, an even function of 
the difference (t t). This may be readily comprehended by considerations 
analogous to thobe ofthe derivations (3.136) - (3.141)of the preceding section. 


Let us make the same calculations by the method of impulse characteristics. 
For brevity we shall restrict ourselves to the case of stationary random processes 
at the system's outputs. 


We shall uge expression (3.32) for our calculations, taking in it n = 2, and, 
taking into account the uncorrelated nature of the applied nput, we shall give it 
a form analogous to (3.131). Then we shall obtain: 


h (3.153) 
met D, tyes fa (t, —¢) be (tg —t) dt, 


where ty > ta. -œ 


By analogy to (3.119), the impulse characteristics o` both circuits are expressed 
as follows: 


t O) = aemet; ta (ft) = aem“. (3.154) 


Now the computation of the integral of (3.153) gives: 


ty 
mo" 1,2) (ty t) = S f are st age“ dt = (3. 155) 





8198 Se-alt.-t) 
at oy 


i.e., a particular case of expression (3.148) for the stead: state. 
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To conclude this section let us consider the calculation of the mixed moment 
by the spectral method. Finding it amounta to computing the integral of (3.100) 
wherein, in this cage, 





Ky (jo) = RiR ’ (3.156) 
Ky (Ju) = rey re (3.157) 
Pa, (~) =+ Fz, (w) = Fon (Jo) = 0. (3.153) 


Expression (3.100) assumes now the form 


mien "D (2) = “tf Ki (Jo) Ka Ju) Fa, (0) et du = nike 


ai sf eins da 3188 Se-s 
2 e 


z Ta — Jolt je mFas 


However if t, > w then, keeping in mind the remark concerning equation 
(3.100), we have: 


mer Dez) + f Ki, (jo) Ka( jw) Fe, (w) ed dz = 


+ 
ans 7 J? du n e 
=- w J (a, je)a — jo) a F ag Sest. (3.160) 


Thus, the stated problem has been solved by each of the three methods set 
forth in this chapter. 


$ 18. The Problem of Two RC Circuits with a Common Output 


The electric circuit in this section and the designation of its parameters 
are shown in Figure 7. The left and right pairs of terminals will be regarded 
as the two inputs, points a and b as the output. Random voltages of equal mean 


squares CH are applied across the inputs, Each of these voltages is stationary and un- 


correlated, the second one being obtained from the first one by the introduction of 
a delay time T. 


A a A 





Figure 7. Two RC circuits with a common output 


By analogy to (3.128) we can express the second moment of each of the input 
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voltages as: 
my”) (ty t) = S-38 (fy — t). (3.161) 


The mixed moment of both input voltages is determined as follows: 


mE? (ty, t) =S- èll ~-t, — T), (3.162) 


where the voltage at the first input is taken at the tim: t, and the voltage atthe 


1 
second one -at to- The objects of investigation are th: statistical properties 


of the output voltage. 


We shall first carry out the analysis of the random process by the method of 
stochastic differential equations. The corresponding differential equation has the 
form: 

du, 
mr) + (a, + 89) bon = Qin, FaMi (3.163) 





where 
1. eat at 
“RC ATRE (3.164) 


Equation (3.163) of the given specific problem corresponds to the system of 
equations (3.22) in the general case. In our cage the number of inputs m = 2, the 
number of outputs n = 1, the output reference number ass imed the fixed value i = 1, 
the input reference number has two possible values j = 1, 2. Since the system has 
degenerated into a single equation, one should take k = 1. 


In the differential equation of type (3.24) enter for the second moment of the 
response the operators; 


Ah nF + +m) ANP? =F + (a+ an) (3.165) 
BY = BMY = BO a, (= =D; (3.166) 
BY) = Bi = Bev =a, i =J = 2). (3.167) 


In view of expressions (3.165), (3.166) and 3.167) w2 shall write the mentioned 
differential equation thus: 


[are +a)|[¥ + +a] me (t,t) = 
= (af +) mr (ty, t) + aaam t? C, ty). (3.168) 


We shall subject both sides of equation (3.168) to a ‘louble Laplace transfor- 


mation with respect to the variables ty and tz We shall ‘ake into account expres- 


sion (3.134) and also the fact that a computation similar to (3.133) yields: 


S PT 
AFRA (3.169) 


The double transform of the second moment of rı sponse can then be ex- 
pressed as follows: 





my” (Pi, Ps) = 
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out) 1 
a ee pei th ata Pat 





+2 a ePT eee a N 
1% ET (Pi + ar t ae) Pat a + ag)’ (3.170) 


Assuming for definiteness t > to and applying the inverse transformation, we 
obtain for t 15 T 


S 
mE ys ta) me y pag Ct ables htwtt- tenth], (3.171) 


for t >T 
Ss 
mont) (t,t) = Carn la: + a; + 2a,a,e -(* +T] x 
K fewest 4) 9 fe, tee +b), (3.172) 


Let us examine expressions (3.171) and (3.172). We note first that at ti = ty 
(out) 
= 0 we have My te 


input voltages has no effect. One can easily convince oneself that (3.171) is the sum 
of the moments of output voltages resulting from each of the input voltages separate- 
ly. The point is that in this case the second input is not yet affected by the fluctu- 
ation impulses correlated with the voltage impulses at the first input. At the time 
t = T the correlation appears and the second moment undergoes a jump-like in- 


crease. The shorter the delay time T, the higher the relative value of this increase. 


ty) = 0. Further, for ti < T the correlation between the 


If t, = œ and t, -+ 00, while the time interval t = |f}, —¢,| remains finite, ex- 


1 2 
pression (3.172) assumes the form corresponding to a stationary random proccss: 


out) - = 
m$ (1) = nra + a3+2a, aze (+0)? e atat, (3.173) 
If T = 0, we obtain from (3.173) 
a (1) = dia + ay) Se- sae, (8.174) 
which coincides with the result for one circuit with the parameter a == aa. 
In fact, under these conditions we can connect in parallel the two inputs of the cir- 
cuit, thus uniting the two circuits into one. ag 
We shall carry out the same calculations by the method of impulse character- 
istics. For making the derivations shorter we shall examine only a stationary ran- 
dom process at the output of the system. 


The impulse characteristics of the system are expressed as follows: 


bi (H) Saye tee, (3.175) 


big (f) = aget, (3.176) 


The relations (3.175) and (3.176) can be easily obtained from differential equa- 
tion (3.163), taking one of the inputs asa unit impulse and the other equal to zero. 


We shall use expression (3.39) to calculate the second momentof response. Since 


only a stationary random process is considered, we shall take the lower integration 
limits of this expression equal to — œ. If the moments are of the form (3.161) 
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and (3.162) one can carry out derivations similar to those preceding formula (3.131) 
and simplify (3.39) as follows: 


mE Ht) =S [i futon tN dtt 


h h 
+ finada HT- Ade f tute +708 alte de 


fe 
+ J bil; — D tlta — i) dt < (3.177) (3.177) 


Substituting the values of $, (f) and €,3(f) in (3.177. and integrating, 
we obtain: 
(out) a sS 
mo GD Ferran X 
x (ai + aj Qa,ayge> (itm) 7} em (ater (t-t), (3.178) 


i.e., a result which coincides with (3.173). 


In conclusion we shall show how to obtain expreasicn (3.178) by the spectral 
method. In accordance with (3.135)the spectral densities of the applied inputs will 


be 


sS 
Fin, (w) = Fin (9) Ge (3.179) 
The mutual spectral density of the inputs in (3.96) is in our cage: 
+0 
l Pe S je 
Pa = t f S04 re a= Sh, (3.180) 


The transfer ratios of the system are given by 
Paai (3.181) 
KuUe)= a Fe, Fie" 


Ku (je) = TEI (3.182) 


The application of formula (3.97) gives now the following result: 
= Í Epp a E cosendo + 
thane S cosurda+ 
+4 erime arte arti ari 


s lat + a+ 2a, e (+a) r] x 


x S gh? gin de = 
SX em (214%) *, (3.183) 


Ta, +4) 2g 


which is identical with equation (3. 173). 
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Chapter Four 


SOME LINEAR PROBLEMS IN THE THEORY OF RANDOM PROCESSES 


$19. One-Dimensional Brownian Motion 


In the absence of an external force field the one-dimensional Brownian motion 
of a particle is described, in accordance with the second law of Newton,by the fol- 
lowing differential equation 


mE+ro=s. (4.1) 


where m and v are, respectively, the mass and the velocity of the particle, r is the 
coefficient of friction encountered by the particle, f(t) is the projection of the total 
force acting on the particle, as a result of molecular motion, on the direction along 
which the motion of the particle is considered. 


Equation (4.1) can also be expressed differently: 
dv 
q+ ev = elt), (4.2) 


where the coefficient ais defined by Stokes's law: 


— bray 
Sa . (4. 3) 


Here a is the radius of the particle and q is the vis@esity coefficient of the 
surrounding fluid. 


Equation (4.2) is called the Langevin equation. 


Since the function g(t) varies much more rapidly than the particle velocity, 
this function can be considered as uncorrelated. 


Equation (4.2) is entirely analogous to the equation (3.102), which describes a 
random excitation of an RC circuit. Consequently, the already obtained result (3, 109)can 
be used, In this connection one ought to take into account that because of the 
noncorrelation of the input it must be assumed that 8-»0o, and to note equa- 
tion (3.124), as well as the fact that the factor a of the equation (3.102) is missing 
in the right-hand side of (4. 2). Having made these allowances, the second moment 
of the particle velocity can be written as follows: for ty >te 


mo (t,, ty) = = Je~st.-&) — e-e hth] = Sena, sh ass, (4.4) 
for t, >t, 
mS (ty, te) = S feet) — gett ty = Se-ch sh at, (4.5) 
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Under steady-state fluctuation conditions the mea1 square velocity of the 
particle will be 
af -5 (4.6) 


Ja’ 
by analogy with (3.125). 
For the determination of the quantity S we can use tie proposition of statistical 
physics, which states that a system in a state of stationary thermal motion has a 


mean energy of kT/2 for each degree of freedom [equipart tiontheorem]. In our case 
2 


mo, 1 
73-7 kT. (4.7) 
This relation in conjunction with (4.6) gives 
S= 2a = atl, (4.8) 
To avoid misunderstandings, let us recall that the function g(t) of (4.2) is equal 
to 
gin =i, (4.9) 


Therefore, the quantity S, which characterizes the ntensity of molecular agi- 
tation, depends not only on the parameters of the liquid bit also on the mass of the 
particle. 


If at the initial moment t=0 the particle was at the origin of the coordinates, 
(x= 0), then at the time t its position will be 


t 
x(t) = femae. (4.10) 
0 


Expression (4.10) is an integral transformation of tie random function v(t). 
This permits to obtain the mean square of the coordinate at the moment t in the 
following way 


t t 
a= f fm, tı) dt, dta. (4.11) 
vu 


Let us calculate the inner integral of (4. 11): 
t ty t 
(ey = S -et S,. at = 
m: (fit) dt, = eon shat, dt, + se st sh at, dt, = 
v A 


= Š (1 — e-h — e*t shat). (4.12) 

z 
Now the calculation of the outer integral of (4.11) gives the following result: 
ob = ph [2at—3 + 4e- ee, (4.13) 


The obtained expression shows that the coordinate »({t) of the particle is essen- 
tially a nonstationary random function, Att=0 we have, as expected, ae 0. 
Will increasing t the mean square 0% grows indefinitel;, This is explained by 
the absence of a returning force. 


If the time t is sufficiently long(zf>> 1), then, in tae braces of(4,.13)all 
terms except the first can be neglected, and the formula ‘4, 13) reduces to 
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(4.14) 


With the use of previously obtained expressions (4.3) and (4.8) for the quantities 
2 and S we can finally write 


2 aT 
r= aa A (4.15) 


This result was first obtained by other methods, by A. Einstein. The experi- 
mental check by J. Perrin gave satisfactory agreement, 


$ 20. Thermal Noise in Electric Circuits 


It is known that a random thermal motion in the conductors of any electric cir- 
cuit gives rise to fluctuating currents and voltages, which are often called noise. 
For calculation of the correlation functions of these fluctuations,any of the methods 
considered in the previous chapter can be used. But any one of them must be sup- 
plemented by a method for calculating the intensity of the fluctuations. 





Figure 8. Elementary RC circuit 


Let us consider the intensity of electrical fluctuations in an elementary RC circuit 
(Figure 8). We shall assume equipartition, as in the foregoing section, i.e., 
that in any system which is in thermal motion the mean energy of fluctuation 
is equal to kT/2 for each degree of freedom. In our case the state of the system 
is completely characterized by a single coordinate: the voltage u across the circuit. We 
have, therefore: 

Ce? = | RT, (4.16) 


where 2 is the mean square of the voltage across the circuit. 


From (4.6) we obtzin: 


2 aT 
Opu = € z (4. 17) 

The fluctuating voltage u can be consicered as originating from an equivalent 
generator with a random electromotive force e(t), which has been connected in series 
with the circuit (Figure 9). Since the thermal motion in the conductors is extremely rapid 
the electromotive force e(t) is considered as uncorrelated. 


For an uncorrelated random excitation of an RC circuit we obtained in § 15 the 
equation (3.118), which we shall write here in the following form: 


2 ! 
Ou = ope S- (4.18) 
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From (4.17) and (4.18) we obtain 
S =2kTR. (4.19) 


The spectral density of the electromotive force e(t) is, in accordance with 
(3.135), equal to 


F(o) = Zeer. (4.20) 





Figure 9. Simulation of electrical fluctations 
in an RC circuit by introduct:.on of 
an equivalent uncorrelated electro- 
motive force 


Formula (4.20) for the spectral density of a fluctuat:ng electromotive force 
was first obtained by H. Nyquist /12/ . The substance of Nyquist's considerations 
is algo expounded in the book of S. Goldman /13/ . The analysis given by these two 
authors is considerably more complicated than ours. But :t ig more rigorous since, 
unlike us, they do not make the a priori assumption of incorrelatedness of the 
random electromotive force. 


Let us clarify which of the elements of the RC circuit (Figure 8) is the 
source of fluctuations. We initially assume that the fluctuations are generated in the 
resistance as well as in the capacitor. Then it can be said that the random electro- 
motive force e(t) (Figure 9) can be considered as the sum of two electromotive forces 
e (t) and e(t), of which the first corresponds to fluctuatio:is generated in the 
resistance, andthe second to the fluctuations originating in the capacitor (Figure 10). 


As the capacitor is a reactance, the power produced init by the electromotive 
force of the resistance is zero, i.e., the resistance does not transmit its thermal 
motion to the capacitor. Therefore, the assumption of a ncnvanishing fluctuation 
electromotive forece ec (t) in the capacitor, leads to the consequence that the 


resistance is continuously heated by fluctuation currents generated in the capa- 
citor, i.e., its temperature must increase beyond all limits. This consequence 
contradicts the law of energy conservation. The circum itance that through the 
resistance R flows its own noise current does not lead tò a change in its tem- 
perature, as the energy of this current is taken from the «nergy of thermal motion 
and is transformed back into heat. 


Thus, the only source of the fluctuations is the resistanc:. The same considera- 
tions could be repeated after replacing the capacitance by an inductance. Thus, 
reactive components do not produce noise . These resulte can be applied to any 
resistances and reactances in a composite electric circuit. 
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Figure 10. Resolution of the equivalent fluctuating 
electromotive force into two components 


§ 21. Thermal Noise in an Electric Oscillation Circuit 


The above results can be used for calculating the thermal noise voltage in an 
electric oscillation circuit (Figure 11). According to what has been said in the fore- 
going section, the only source of the fluctuations in the circuit is the resistance r. 
The "noisy" resistance r can be represented as a "noiseless" resistance of the same 
value in series with an uncorrelated noise electromotive force of spectral density 


F (wi) = RTT. (4.21) 


Then the equivalentcircuitof Figure 12 can be used for the calculation of the 
noige voltage u. 





Figure 11. Electric oscillation circuit 


The transfer ratio of the noise electromotive force is 


use sà (4.22) 


KUD= TH jel + E ~ Lay tae" 


where w = 1/ V LC is the natural frequency of the circuit and 2 =r/2L the damping factor 


The square of the modulus of the transfer ratio will be: 
|K(ju)|? “i (4,23) 
‘w) |*§ = —— r. i 
(o? — w?)? + data? 
The spectral density of the noise voltage u in the circuit is determined from 
(4.21) and (4.23) in the following way: 

a 

2RT a 
F hiy a : (4.24) 

u(0) z (ot — wo)? + dato? 
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Let us examine the structure of this equation. First. we shall determine 
the effective component of the equivalent resistance of the circuit (Figure 11): 


L Re [Ort fol) jw te (4.25 
R(w) = Re Satieel= "(ol aan 


By combining the equations (4.24) and (4. 25) we obtain for the first of them 
the following simple form: 


Fy(w) = 2 kT R(w). (4.26) 


A comparison of (4.26) with (4.20) shows that the expression (4.20) can be 
used in the calculation of the noise electromotive force as the effective resistance 
as well as the effective component of the complex resistan: e. 


If the Q-factor (Q= m, L/r) is sufficiently high, the effective component R (w) 
of the equivalent resistance of the circuit and the spectral density Fa (w) of the noise 


voltage, which is proportional to R(w#), have a sharp maximum at a frequency prac- 
tically equal to the natural frequency wy of the circuit. This maximum is the sharper 


the higher the Q-factor. Thus,with a high Q, the main part of the fluctuation 
energy is concentrated in a narrow frequency band, cen‘ered about about the natural 
frequency w of the circuit. This means, that under the indicated conditions the 


fluctuations resemble harmonic oscillations having the nitural frequency of the 
circuit, 


Using expressions (3.63) and (4.24) let us compute the second moment of the 
noise voltage across the circuit: 


ms" (1) = J Fu (w) cos wt dw = 


L BT d wet S cos wt dw 
(e? 


— w2)? + datat Fs 
RT 
He (cos wt + = sin wt), (4.27) 
where 
w = w? — 22, 


The mean square of this voltage can be found by assu ning fT =0 in (4.27) 


u= -r (4.28) 





Figure 12. Simulation of electrical fluctuations in an oscillation circuit 
by an equivalent uncorrelated electromotive force 
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If only the last result were required, it could 
Peres ye ist q y have been obtained immediately 


1 
kT. (4.29) 


which is known from the foregoing sections. 


Let us continue the examination of equation (4,27), Noting that the quantities 
aand w are connected by the obvious relation 


l wo 


a = -7 


7? Q (4.30) 


and assuming that the Q-factor of the circuit is sufficiently high (generally Q > 100), 
(4, 27) can be simplified to 


mS") (2) = ope- "cos wot. (4.31) 


In order to grasp the meaning of this equation we shall consider the 
harmonic oscillation 


x(t) = X cos (ws + P). (4.32) 


where ọ is the random initial phase, all values of which are equally probable. Under 
this assumption the second moment of the oscillation can be calculated as follows: 


mE (ty, t) = M ix (4) x(t) = 
= M [X cos (w, + 9) - COS (wef, + p) = 


= 5 X? {M [cos wo (f3 — £1) + M leos (wo (f, + t) + 29)]} 
(4.33) 


and finally 


mE (t, t) = mP (2) = + X? COS wot = 03 COS wot, (4.34) 


where c= (|4,—4,|: oly 


As pointed out above, the voltage fluctuations in a circuit with a high Q-factor 
are nearly harmonic oscillations, They contain, however, a certain random 
element. The larger the random element the weaker is, at fixed t, the statistical 
dependence and the smaller its quantitative measure-the second moment mg7(t).This 
randomness of the oscillatory process can be accounted for by introducing into the 
equation (4,34) the factor $(t), which is equal to unity at t= 0, and decreases with 
increasing t. Thus we obtain 


mS7(x) = 03 Y(t) cos wot, (4.35) 
Equation (4.35) is typical of those stationary random process which closely 
resemble harmonic oscillations. In particular, of such a form is the second moment 
of the fluctuations at the output of any selective system, i.e., asystem with sufficient- 
tly sharp resonance properties, which is subjected to an uncorrelated input. The 
form of the function $(t) is determined by the structure of the system. 
For a single oscillation circuit we obtain from (4, 31): 


po =e-™. (4.36) 
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It follows from previous considerations that the damping factor a entering 
(4. 36) characterizes the degree of randomness of the fluctvations. 


The character of voltage fluctuations in an oscillation circuit or at the output 
of any selective system can be easily found from purely qvalitative considerations, 
by following the method of impulse characteristics. As already remarked in $ 13, 
an uncorrelated input can be considered as a succession of ‘mpulse functions with un- 
correlated areas, following immediately upon each other. 4 single needle-shaped 
input pulse imparts to the circuit a certain store of ene.-gy, initiating damped 
oscillations of frequencyw,. The resulting process is the sum of infinitely 
many elementary oscillations of this kind. The oscillations from previous pulses are 
attenuated, but the energy of the fluctuations is replenished by newly produced oscil- 
lations. The sum of any number of oscillations of same frequency wọ is also an 
oscillation of the same frequency wọ. 


Since the initial amplitudes of the continually arising elementary oscillation 
processes change at random from one process to another, the amplitude and the phase 
of the resulting oscillation are also continually fluctuating, i.e., they are random 
functions of time. 


If the statistical properties of the instantaneous noise voltage are known, the 
statistical properties of the random amplitudes and phases >f the fluctuation process 
can be investigated. But this necessitates the use of nonlin:ar transformations of 
random functions, which we have not yet considered. We postpone, therefore, the 
further consideration of the above problem to the sixth chapter. 


§ 22. Thermal Motion of a Galvanometer 


When working with highly sensitive galvanometers one}! as to take into account 


that the moving system of the instrument is in a state o” incessant agitation, 
analogous to the Brownian motion, which renders very dfficult the measurement 
of very small currents. This motion is caused, on the one hand, by the molecular 
motion of the air surrounding the system, and on the other and by the electric fluc- 
tuation currents in the galvanometer coil. 


We shall consider first the random fluctuations of the moving system of the 
instrument, with the circuitofitscoil open. Then no current flows through the coil 
and the behavior of the galvanometer can be described by the usual differential equa- 
tion of torsional oscillations 


G+ r$- +D = MO, (4.37) 


where @ is the deflection angle of the moving system of the instrument, I - its moment 

of inertia, r - the coefficient of friction, D - the rigidity of suspension, and M(t) - the 
random torque exerted on the moving system by the molecu: ar motion of the sur- 
rounding medium. It is natural to consider the random function M (t) as uncorrelated. 
We shall examine only stationary fluctuation conditions, using the method of im- 

pulse characteristics. 


Assuming the right-hand side of equation (4.37) to be ı unit impulse function and 
applying to both sides of this equation the Laplace transforriation, we obtain the fol- 
lowing expression, representing the impulse characteristic of the galvanometer: 


= 1 
eras eee (4.38) 
oC) pl+ pr+D 


and by introducing 2= r/2I - the coefficient of damping, ® = V Dil -the natural fre- 
quency of the moving system, B? = a? — ws, we obtain 


ry 1 
90) = Ta (4. 39) 
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In the following, only the aperiodic case of the motion will be considered, 
when a? > =. i.e., P >O. Then the inverse transformation of (4. 39) gives 


0(t) = + e-*tsh Be. (4.40) 


For the calculation of the second moment of the angle § we use equation 
(3.132). Substituting in its right-hand side the expression (4.40) for the impulse char- 
acteristic of the instrument and integrating, we obtain 


ms (2) = pS et ch Bt + g shB] TI). (4.41) 


Assuming in (4.41) t=0, we obtain the mean square of the angle 9: 


2 S 
a= 7p: (4.42) 


To find the value of the unknown ‘quantity S we shall apply the same method as 
in foregoing sections. The component of the mean fluctuation energy with respect 
to the coordinate bis 


+ Då = 5 kT. (4.43) 


Combining (4. 42) and (4. 43) we get 


S=2kTr. (4.44) 


The obtained result calls for two comments. First, its similarity to the 
formula (4.19), which was obtained in § 20 for the noise electromotive force of the 
resistance, shouldbe noted. Second, we note the double role played by the medium 
which surrounds the moving system of the instrument. On the one hand the medium 
opposes the motion with a moment proportional to the coefficient of friction r, on the 
other hand it causes the random torque M(t), whose value S is proportional to the same 
coefficient r. 


In view of equation (4.44) we can put relation (4.41) into its final form: 


m9 (= AZ e-ti (ch pe + 7 shBlt)). (4.45) 


As a check on this result let us calculate the component of the mean energy with 
respect to the other independent coordinate of the moving system - the angular rota- 
tion velocity w=d/dt. As was shown in $ 7, on differentiating a random function, 
its second moment is differentiated twice. If the differentiated function is stationary, 
as in the present case, this differentiation is done with respect to the time interval 
t and the obtained value is taken with the opposite sign (formula (2.41)). Thus: 


Pm?) (1) 
() poy 2 
mz (t) =- ` (4.46) 
Differentiating twice and then puttingt=0 we obtain: 
2 aT (4.47) 
ay = T . 


The mean energy of fluctuation along the coordinate w is, as expected, 


en 


kT. (4.48) 


vl = 


+h 





Figure 13. Moving system ofa galanometer 
in a magnetic field 


So far we assumed that the circuit of the galvanometer coil is open. Now let 
us suppose this circuit to be closed by the resistance R. For simplicity we shall as- 
sume R to be so large, that the resistance of the coil and the action of its self- 
induced electromotive force can be neglected. First we stall examine the motion 
of the moving system of the instrument as caused by the action of the surrounding 
medium. In that case the closing of the circuit should be ccnsidered only as equiva- 
lent to the appearance of an additional retarding moment, cused by the current in- 
duced in the coil. 


Let us calculate the electromotive force induced in the coil by the rotation of 
the moving system. For this we shall use Figure 13. The ‘ield in the gap of the 
magnetic circuit of the instrument can be considered as rad:al, in a first approxima- 
tion. In addition, it can be assumed that on rotation of the «oil, its sides parallel to 
the axes move in a uniform magnetic field, Let v be the velocity of this movement, 
nthe number of windings of the coil, and B the magnetic incuction in the gap (other 
designations are shown in the figure). Then the induced ele :tromotive force is 


e = 2Blvn. (4.49) 
Since the velocity v is expressed by the angular velocity of rotation as follows: 


b b 2 
per > z 4.50 
v=voz7 $7 t’ ( ) 


formula (4.49) can be given its final form: 


dð 


e=A T 


(4.51) 
where 


A = Bibn. (4.52) 


Under the above assumptions this electromotive force generates in the 
circuit of the instrument the current 


jot At (4.53) 


Interacting with the magnetic field,the current i generates the retarding torque 


M.a = Bilnb = Al = E L 


R a’ (4.54) 


By introducing into the left-hand side of differential equation (4.37) the addition- 
al retarding torque (4.54), we obtain: 


do , dü 
le te ap t+ t= MO. (4.55) 


Here 


pepe (4.56) 
T f 


The statistical characteristics of the right side of (4.37), as expressed by the 
formula (4.44), remains unchanged. Therefore, without further calculation, we 
can use the ready result (4.41), by replacing it in r by r' and retaining the previous 
value of S, as expressed by formula (4.44). Thus 


@, AT f ar f , 
ms (8) = Te ench pt gy sh 3 It], (4.57) 
where 
r’ 
=g = Vo o, (4.58) 


For the mean square of the fluctuations of the angle §, which are produced by the 
molecular motion of the medium surrounding the galvanometer, we obtain: 
2. kT r 
ARD ae (4.59) 
The mean square of the fluctuations due to thermal motion of the medium is 
thus seen to decrease owing to the retarding torque of the forces on the induced 
current, 


Let us now examine the random oscillations of the angle 6, which are produced 
by the electromotive force e ,(t), generated in the resistance R. The random torque, 
generated by the action of this electromotive force, is equal to 


M,() = AL, (= 4 e (b. (4.60) 


In accordance with $ 20 this electromotive force can be considered as uncor- 
related, and its magnitude Se, is expressed by formula (4.19). Therefore, the torque 


M,(t) must be also considered as uncorrelated, and its magnitude Sy, is determined 
by: 
A? 


Sm = A Sa SORT 
R R (4.61) 


The random function 9, satisfies the differential equation (4.55), in 


the right-hand side of which the torque M,(t) replaces the torque M(t). Therefore, 
the expression (4.41) in which r' replaces r and the quantity S is defined by formula 
(4.61) is valid for the second moment of the random angle 4,. Thus, 


mm? (t) = as e-win (ch Bit F sh Bp’ t]). (4.62) 
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The mean square of the fluctuations of the angle 6, is equal to 


2 RTA? 


a, = DR: (4.63) 


Th2 random angles 9, and 0, are statistically independent of each other. There- 
fore the mean square of the resulting angle — 6, + 6,is obtained from (4.56), (4.59) 
and (4.63) as follows 


2 42 2 Tr RTA? _ RT At) RT 
08 = o, + 98, = py DR PtT (4.64) 


i.e., the mean energy of the fluctuations along the coordinate 0is, as before, equal 
to 
Ds, 1 ar 
Ha pes 





aT, (4.65) 


nli- 


Thus, closing of circuit of the galvanometer has no influence on the 
mean square of the fluctuations of the angle §. In other words, whatever the mech- 
anism of the transfer of thermal motion to the system, the mean square of its fluctu- 
ations along each of the independent coordinates remains unchanged. 


From this constancy of the mean square of fluctuations in no way follows 
that the character of the random motion of the moving system of the instrument is 
independent of the quantity 2’. This can be easily seen by considering the 
spectral density of fluctuations. Their second moment car be represented as follows, 
in accordance with the results obtained above: 


mi) (x) = ofe- 2 It! (ch B+ a sh piel). (4.66) 


By substituting this expression in formula (3.62) and integrating, we obtain after 
some simple transformations: 


ee l 
Fy(w) = Za OT EL ea (4.67) 


Introducing the dimensionless frequency = w/w, and the damping factor 
d= 22°) we obtain finally: 


Rim = 3 < o2@ (t. d). (4.68) 
where 
[i 
9(§ d) = GT pee - (4.69) 


The function 7( §,d) characterizes the dependence of the spectral density of 
fluctuations on the frequency. The graph of this dependenc? is shown in Figure 14. 
It can be seen from it that at sufficiently small damping factors d(galvanometer 
circuit open, friction coefficient r small) the spectral dens ty of fluctuations has a 
sharp maximum in the range of frequencies near the natural frequency of the moving 
system. In other words, the character of fluctuations is similar to the character of 
harmonic oscillations of frequency ,. On the other hanc, for large values of the 


damping factor, the fluctuations have a highly random char.icter. 


The thermal motion of measuring instruments and its influence on the accuracy 
of measurement are considered in detail in the monograph »y V.L. Granovskii /14/. 





Figure 14. Spectral density of galvanometer fluctuations 
as a function of frequency 


§ 23. The Passage of Irregular Telegraph Signals through a Linear Filter 


Let us consider a telegraph signal consisting of a sequence of elements of 
two kinds (so called binary transmission), The simplest element of the signal 
is a rectangular pulse of amplitude A and duration T. We assume that all pulses 
follow directly upon each other, have equal characteristic parameters A and T, but 
can have with equal likelihood positive or negative sign. Moreover, we assume that 
the signs of individual pulses are statistically independent of each other. This is 
the simplest mathematical model of a telegraph signal, which we shall adopt in the 
following considerations. 


Letus find the second moment of sucha signala(t). At r>T, by virtue of the sta- 
tistical independence of the signs of two neighboring, non-simultaneous pulses, we have: 


mY) (1) = Mla (t) -a(t +1) = 0. (4.70) 


Let nowt <T. We choose a certain time moment t, contained by one of 
the pulses. Let us assume that the time interval between the beginning of this pulse 
and the moment t is a random variable, which can have with equal probability any 
value between zero and T. A later moment t +t can fall within the same pulse 
or the next one. The first case takes place if 0< t< T—t. We have 


Mla(t)- a(t +2) = A. (4.171) 


T—¢ z 
The probability of the first case in equal to —ọp~ = I —- y» Hence if 
T-t <t < T, the time moment t++ is contained by the next pulse. Then 


MĮat)at +1) =0. (4.72) 


The probability of the second case is t/T. Averaging the product a(t). a(t+ t) 
over all possible initial positions of the pulse, we have : 


mP = A (1 —4)+0 4 =A(1 —+)- (4.73) 


Since all the pulses are completely equivalent, this result is valid for the inclu- 
sion of the time moment t within any of the pulses. Thus, (4,73) is the 
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sought-for expression for the second moment of the telegraph signal. The mean 
square of the signal ordinate is 3? = A2, Therefore, formula (4.73) can be written 
as follows: 


m{(2)() = 3? (! = F): (4.74) 


mt] 


j 
| 


? 
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Figure 15. Graph of the second moment 
of an irregular telegraph signal 


The graph corresponding to equation (4.74) is shown in Figure 15, Using 
the formula (3.62) we find the spectral density of the signal 


T 
Pw) =2 f a (1 — F) cosord = 


2 _,1—cos 
= for, (4.75) 


w? 


The spectrum of the signal is shown in Figure 16. We have now obtained the 
necessary statistical characteristics of the signal and can proceed to the analysis of 
its action on a linear filter. 

o7 
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Figure 16. Graph of the spectral density of 
an irregular telegraph signal 


Let us consider an RC circuit (Figure 2) as the simplest filter. Its behav- 
ior is described by the differential equation (3.102). 


The filter causes distortions of the signal, i.e. , the-e appears a certain error 
e (/)= Bou (f)— Uin (t), (4.76) 


which is, as the two terms of the right-hand side of (4.76), :¿ random time-function. 
We can use the mean square o? of the error e(t) as a measure of the distortion, 


Let us write the differential equation (3.102) as follows: 





d (kou ~un) a 
i FA tor nd Sm r (4.77) 
or, differently, 
dı düin 
=: = 4.78 
dt di * ( ) 
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We shall use the spectral method. Then we have to calculate the frequency 
characteristic of the filter error. For this we assume 


uin = Ù nett, e = Eej“! (4.79) 


and substitute these values into the equation. We get: 


JoE + aE = — join, (4.80) 
whence 
=- 
K (jo) = i. ET (4.81) 


The square of the modulus of the frequency characteristic of the error is 
equal to 


w? 
IK (jo)? = aya: (4.82) 


The spectral density of the error is expressed as: 





207 |—cosal 
P,(w) = Pin (0) + |K (Jo) P= > “ayer: (4.83) 


The mean square of the error is: 


20? 1 — cos oT 
d= f de= aT E ene le, (4.84) 
Q 


The relative root mean square error is defined as follows: 


qettay 4e). (4.85) 


We introduce the pass band âfo.1 of the circuit, by which we shall understand 


the frequency band in which the modulus of its transfer ratio (3.130) is not less 
than 1 V2. It can be readily seen that to the upper limit of this band corresponds the 
angular frequency wọ, = 3, whence we have: 





for =R: (4.86) 


Now, expression (4.85) can be brought into following form: 


1 = 
1=V maqar (te er"). (4.87) 


If the pass band of the filter Af, , = 0, then the relative error becomes7= 1, 


i.e. , the signal is not reproduced at the output of the circuit. At Afo, —> 0o we have 


N— 0, i.e., the error tends to zero, The dependence of the relative error 
on the product Af, ,7 is shown in Table 1. 


Table 1 





At Afy.,T > 0.5 the value of 4 can be calculated by the approximate formula 


1 


= VEF . (4.88) 
n e? 


with an error of less than 2%. 


Taking = 0.3 (about 10% of the signal power), we obtain from (4.88): 


Afy.:T 1.8. (4.89) 


It is generally assumed that, for a satisfactory reproduction of the form of a 
telegraph signal with continually alternating signs of the pulses, the filter must let 
through the third harmonic of the basic frequency of the signal. The basic frequency 
of such a signal is equal to 1/2T. Thus we obtain: 


3a = Alor or Afo =15, (4.90) 
i.e., a result which is close to (4. 89). 


Let us now consider the mean square error of such a filter, which results 
if, in addition to the useful signal, an uncorrelated noise of spectral density Fy is 
applied to its input. Since the signal and the noise represent processes which are 
statistically independent of each other, the mean square o2, of the resulting error 
can be expressed, in view of (3.118), (3.135) and (4.84) as follows: 


2 8? -eT i F 
s= zr le +52 ne (4.91) 


whence we obtain for the relative root mean square error 


waZ=y (1 —e-*T)+aTA, (4.92) 
where 
jain (4.93) 
er a ` 


Let us trace the connection between the resulting er-or and the pass band 
width, i.e., the quantity aT. If the pass band is narrow, the main component os 
the error is produced by the distortions, caused by the filter, asthe mean square of 
the noise at the output is small. Conversely, ifthe pass ban iis wide, the distortions 
mentioned above are insignificant, but the mean square of tie noise voltage at the out- 
put increases in proportion to the band width. 


From these considerations one could expect that it i; possible, under certain 
conditions, to realize a most advantageous pass band, whic: would ensure a minimum 


value of error. Referring to the extremal values of expression (4.92), the mini- 
mum of the relative error 9, can be readily obtained for the condition 


ia SeT. 
AE ui pa 


From this expression Table 2 has been constructed ior optimal values of aT. 


Table 2 
(@Mopt | 0 | 1 | 2 | 3 | 5 


1.00 | 0.26 | 0.15 | 0.089 | 0.038 
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At (2 T) opt >5 one can set, with an error of less than 4%: 





= ant, , (4.95) 
hi 
w encer can be con 
1 
(AT on = Va: (4.96) 


It can be concluded from these results that the higher the relative noise level 
A at the input of the filter,the narrower the optimal pass band. At A> 1 no optimal 
band exists. 


In view of relation (4.89), to which corresponds2T= 11.3, we conclude that 
large values of 2T, for which formula (4.96) is valid, are of practical interest. Sub- 
stituting the value for2T obtained from (4.96) into (4.92), and taking into consider- 
ation that for2T>5 the exponential term can be neglected, we obtain: 


4 
Tu min = V 2 VA. (4.97) 


Given the required value of humin we can find from this expression the corres- 
ponding value of A. With known level of the noise and the desired duration of the sig- 
nal, i.e., the speed of transmission, this permits to obtain, with the help of (4.93), 
the necessary level of the useful signal at the input, and also to find from (4.96) the 
optimal pass band of the filter. 


The foregoing was an example of the analysis of a specific system, from the 
point of view of the root mean square error, suffered in it bya signal. The criterion of 
root mean square error was introduced into the theory of random processes by 
A.N. Kolmogorov /15, 16/ and was applied to a number of practical problems by 
N. Wiener /17/. At the present time, this criterion of quality of a system finds an 
ever increasing application, especially in the theory of automatic control /18,19/ . 
But it should be in no way considered universal. This is seen, for instance, from 
the following example. In pulse radar for long distance detection it is desirable 
to be able to detect the weakest reflected pulses against the background of 
fluctuation noises, Here the distortion of the pulse shape at reception is a second- 
ary factor only, as it is first of all important to ensure that the peak value 
of the signal should surpass the noise as much as possible, 


§ 24. The Optimal Filter Problem 


The linear problems of the theory of random processes, as cdnsidered in 
previous sections, are characterized by the following statement of the problem: 
given a specific system and its parameters, as well as a sufficiently complete sta- 
tistical characterization of the input; to calculate the responses of the system to 
this input. These problems may be classified as belonging to the analysis of sys- 
tems under random excitation. 


In all cases, except the one considered in $23, onlythe responseto random 
input was considered, and no conclusions about the desirable values of the para- 
meters of these systen.s were drawn. But from the point of view of classification 
of problems this fact is not essential. 


Besides the problems of analysis, probleme of synthesis of systems subject 
to random excitation are also of great interest to the modern technology of auto- 
matic control. In these problems the structure of the system is not defined, 
and the totality of all possible systems of a certain class (e. g., the class of 
linear systems) is considered. It is required to choose from this class the 
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optimal system (e. g., from the point of view of minimum root inean squareerror), This 
problem was first considered in its general formulation bv A, N. Kolmogorov /16/, 
Results important for practical applications were also ob:ained by N. Wiener /17/. 
A clear and comparatively simple presentation of the corresponding mathematical 
questions was given in the comprehensive article by A.M. “aglom /3/, The same 
problem was considered in the monograph of V. V. Solodovr ikov /19/ and in the later 
works of V. S. Pugachev /20,21,22,23/. We will give here only one simple particular 
example for the synthesis of an optimal system, which ensures minimum root mean 
square error of reproduction for a useful signal of specific shape and uncorrelated 
fluctuations at the input. 


As the signal we take an input with the following properties. The voltage u(t) 
can take either of two discrete values +U and -U. The mean number of changes 
of sign of the signal per unit of time is denoted by n. “he instants of sign change 
are random, and the probability for k sign changes during tie period t follows the 
Poisson distribution 


W (hy = EM e-m, (4.98) 


We remark, that the Poisson law is met with in mary statistical problems, in 
particular in the theory of electron emission by hot cathodes. Ifthe mean number 
of electrons emitted by the cathode in unit timeis n, and the individual electrons are 
equally likely to leave the cathode at any time and do s» independently of each 
other, then the probability for emission of k electrons ty the cathode in time 
t is expressed by the same formula (4.98), 


Let us find the second moment of the signal. The v:ilue of the product uc(t)- 
"u eltt t) is equal to vu, if during the time t the number of sign changes has been even, 
eat equal to -u2 for an uneven number. We have, therefore, 


mS) (x) = UW (0) 4+ W(2) 4+ WA) ...J-- 
—UWII)+ W(3)+ W(d)4 . 


or, substituting the value of W(k) from (4.98) , we obtain 


(4.99) 





mic) (y= Ue (1 — a Tapani. iy -e +... }=U?e-?m, (4.100) 


Let us now formulate our problem. At the input cf the linear electrical 
system acts a stationary random excitation consisting of the signal us and the 
fluctuation noise ui: 


u, (f) = u(t) + u (f). (4.101) 


The second moment of the signal is given by the expression (4.160) and 
the noise is uncorrelated. The voltage u,(t) at the outpu: of the system re- 
produces the signal uc(t) with a certain error 


e(t) = u, () — u. (t), (4.102) 
whose mean square is equal to 


a? = of + o? — 2m2 (0). (4.103) 
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where mio) 


voltage, calculated for the two voltages considered as functions of the same 
time argument. 


(0) is the mixed moment of the input signal voltage and the output 


The mean square error o? is determined, on the one hand by the external in- 
fluences, and on the other hand by the structure and the parameters of the system 
which are unknown. The problem consists in finding the structure and the parameters 
of the linear system which would ensure a minimum mean square error under given 
external conditions. 


Let us set up the explicit expression for the mean square error. The mean 
square of the voltage at the output of the system will be determined by formula (3.30) 
in which we set t = to =0 and the lower integration limits as equal to — .c: 


o o 
g= f faper dr (4.104) 
-00 -00 
By change of integration variables is -xX, t= ~y, equation (4,104) assumes 
the following form: 
w 


ob = f Edy f m y — xE) dx. (4.105) 
v 


u 


For computing the mixed moment m{?¢)() let us write the integral trans- 


formation which determines the response of the system 


o æ 
u, (0) = fanna = f m Ot d. (4.106) 
-0 o 
The moment m*ho) can be considered as the mixed moment of two linear 


transforms of random functions, the first of which being defined by the expression (4.106) 
and the second consisting of the product of the random function u,(t) and unity. Having 
in view the general formula (2.73) we obtain: 


m$ (0) = fm (x) E(x) dx. (4.107) 
v 


where m,1°) 


(0) is the mixed moment of the voltages u(t) and u(t which, because 
of the statistical independence of the signal and the noise, is equal to 


mg’ (x) = mẹ’ (x). (4.108) 


In view of equations (4.105), (4.107) and (4.108), we can now write the mean 
square of the error in the following way: 


oi =o — 2 f mP (x) E(x) dx + 
ve 
+ fio dy f mi (y — x) tx) dx. (4.109) 
o o 


Relation (4.109) is valid for any form of the moment functions of signal 
and noise. Only their statistical independence is necessary, in which case follows 
(4.108) which we used above. 
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The mean square of the error (4.109) depends on tne form of the impulse char- 
acteristic: (x)ofthesyatem. First one has to find such a function€(x), for which the 
quantity 2 has a minimum. Problems of this kind are dealt with by the variational 
calculus, whose methods we shall use. Then, once the iripulse characteristic of the 
optimal system is known, one has to determine the structire and the parameters of 
this system. 


For determination of the impulse characteristic of the optimal system we shall 
proceed in the following way. Let us assume that the minimum of the mean square 
error is obtained for the impulse characteristic E)(x), W= shall replace in the 
expression (4,109) the function —(x) by the function §)(x)+ n(x), where n(x) is 
some function which vanishes for x< 0 and is otherwise arbitrary, and y is a pa- 
rameter independent of x. The indicated operation of the calculus of variations is 
equivalent to anincrease of x by Ax in differential calculus. 


After this substitution it is easily found from (4. .09) that the departure 
[variation] of the mean square error from its minimum @fuals 


Bo! =| — f mP ndx + 
ò 
+ fioa fao- onma + 
ô ò 


+78 fandy f mpy — x) 9 (x) dx. (4.110) 
o o 


The necessary condition for the extremum of the cuantity a is analoguous to 
the corresponding condition for extrema of functions of ore independent variable, 


and it has the form 
[aen] =0 (4.111) 
dy {20 


for an arbitrary function (x). 


By carrying out operation (4.111),we obtain: 


fm?) v9 dx— f ny) dy f mP (y—2) (x) dx=0 (4.112) 
o o o 


or 
œ ao 
TROLS [Po-So aunas] (4.113) 
o o 

Since the functiony(x) is arbitrary, condition (4.1: 3) can be fulfilled only for 
œ 
mP) = f mPy—a)(xde (04y < o0). (4.114) 
° 


Such a condition is necessary for the mean squar? error to be a minimum. It 
can be proved that this condition is also sufficient. 


Equation (4.114) is Fredholm's integral equation of the first kind. For the 
general case it is relatively difficult to obtain its solutio1. Therefore, we terminate 
our calculation of the general case by the obtained result, and turn our attention to 
the specific problem, as formulated at the beginning of tiis section, for which a 
solution can be obtained by simpler means. 

(1) 


The moment m,‘°) is expressed by formula (4.10)). The moment mọ © of 


the input voltage consists, because of the statistical independence of the signal 
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and the noise, of the sum of their moments, which we can write in view of expres- 
sions (3.128) and (4.100), in the following way: 


mi!) (y — x) = U*e-Ply-zt Sè (y — x), (4.115) 


where 3 = 2n. Let us substitute in the integral equation (4.114) the indicated values 
for the moments. Then, in view of 


f Sy — D&G) ax = SH), (4.116) 
we have: o 
Ue-by = Sto(y) + U? f e-P1 v-z (x) dx. (4.117) 


v 


Equation (4.117) is Fredholm’ s integral equation of the second kind. Let us 
apply to both members of this equation the Laplace transformation. The trans- 
form of the definite integral, in the right-hand member of (4.117), is equal to 

a w 


Je dy f e-Plunzik, (x) dx = 
ò o 


= fi (ods fecre-ste-mdy = 


u 
Li ole -9 œ- remayt fe By ze- ma) = 
v 








e-'rg (x) dx — ee J erd = 


lucs 2 = 
= 55 bA — ae A. (4.118) 


Taking (4.118) into account, we obtain the following operational equation: 


ory = Sh + RO pb). (4.119) 
whence 
_ 1+ ht) 
E (p) = VË Ẹ 1 — & (B) 
& (p) = > {1 — &, (B)] ————-. (4,120) 
S p-p- 


From the list of formulas of operational calculus we take the relation 


Be Be i =e =3( —) dt z( 2\e-dt 
Fae och bt —F shot = (1 gah HS) (4.121) 

As the required impulse characteristic has to satisfy the condition {(oo)= 0, 
the function of which (4.120) is a transform cannot contain a term which would 
increase exponentially with increasing argument. Therefore, in (4.121), a must 
equal b, which corresponds in (4, 120) to 


pie — pr 9p 2. (4,122) 


Relation (4.122) permits the elimination of the quantity £03) from the equa- 
tion (4,120) and thus to simplify it as follows: 
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k= l 
S= PTFE ptIYTFE' a 


where 
2U? 
k= + (4. 124) 


Let us recall that the quantity B entering (4. 123) anc (4. 124) is equal to double 
the average number of sign changes per time unit. 


By transforming (4.123) back to the original, we obtain 


AYTTI FRANE, 
]evi+ (4.125) 


Wo =z: =[F 


Thus, the required impulse characteristic of the optimal system has been found. 
Knowing it, the structure and the parameters of the system can be determined. Neg- 
lecting the factor in the square brackets of (4.125) and comparing (4.125) with (3.126), 
we can conclude that the RC circuit with the parameter a equal to 


a=pV TFR. (4.126) 
has the impulse characteristic (4.125). 






ut Youll) 


Figure 17. First circuit of an optimal filter 
for an irregular telegraph signal 
and uncorrelated fluctuatioa noise 


The presence of the above-mentioned factor means that a voltage divider must 
be introduced into the circuit. In this way the circuit of F: gure 17 is obtained. Its 
parameters are to be chosen in accordance with the condit:ons 


Rio RO (4,127) 
Rit R, itk’ 





Thi t a) =eVTFR (4.128) 


Assuming arbitrarily one of the parameters, the two others can easily be 
found from (4.127) and (4.128). Of course, the shown stru::ture of the optimal system 
is not the only one possible. In particular, the same resu t is obtained with the cir- 
cuit of Figure 18. 







CARL touilh 


Figure 18. Second circuit of an optimal filter 
for an irregular telegra»h signal 
and uncorrelated fluctuetion noise 
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The task of the above-considered optimal system was to reproduce, in the 
presence of interference, the input signal with minimum mean square error. In the 
theory of automatic control a more general problem is also considered, where a 
system operating under the same conditions has to perform with minimum of error a 
certain transformation of a signal; the line of reasoning in this case is analogous 
to the foregoing. 


§ 25. Elements of the Theoryof Potential Noise-Stability 


The concept of optimal system, as introduced in the preceding section, is 
in no way universal. In the following a different formulation of the problem is con- 
sidered, in which the concept of optimal system acquires a completely different sense. 
This theory, which will be sketched in the following,was developed by V. A. Kotel'nikov 
124/, 


Let us consider a communication channel under the action of random inter- 
ference. The method of technically realizing the channel is of no importance for the 
following. We shall assume that for the transmission of the information through the 
channel two kinds of signals of equal duration T, denoted by A(t) and B(t) are used. 
An example of such a transmission of information is telegraphy using the Baudot 
code, where one of the signals is switching the current on, and the other is 
the interval or switching on a current of opposite sign. The mode of transmis- 
sion is assumed to be known, i.e., the determinate functions A(t) and B(t) are given. 


The input of the apparatus which is placed at the output of the channel—it will 
be called in the following receiver—is acted upon by X(t), being at any moment the 
sum of one of the two indicated signals and of the interference W(t), which is a 
random function of time. Thus, it is either 


XOH=AOAW,O, (4.129) 
or 
XHA =B NHW: (t), (4.130) 


where W(t) and W,(t) are distinct realizations of the random function W(t). 


When receiving the signal X(t), the receiver must respond to one of the two signals 
A(t) and B(t) which constitute its input. The basis for the response is the com- 
parison between the disturbed signal X(t) and the signals A(t) and K(t), which 
are in some way applied to the receiver. The reaction of the receiver is determined 
by the closeness of X(t) to one of the signals A(t) and B(t). The concept of closeness 
here used is conditioned by internal characteristics of the receiver, 


As the various realizations of the disturbance may vary greatly in form, 
it may happen that the input X(t), containing the signal A(t), is closer to the signal 
B(t). In such a case the response of the receiver will be false. The receiver may 
also respond to an X(t) containing K(t) as if it were the signal A(t). Such errors 
cannot be eliminated in principle, i.e., a certain percentage of response errors 
is unavoidable. The probability of error depends on the internal structure of the re- 
ceiver, i.e, on the concept of closeness of two functions it employs. 


Following V. A. Kotel'nikov we shall call a receiver ideal, for which the proba- 
bility of error is minimum. This constitutes the optimal system, whose properties 
are studied by the theory of potential noise-stability. 


First we have to clarify the concept of closeness which is needed for the re- 


ceiver to be ideal inthe above-mentioned sense. We shall assume that the inter- 
ference is a stationary random function of time, with normal distribution 
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wr 
w(W)= 7E e =, (4.131) 





After receiving the signal X(t) we can say that in he givencase the interference 
had appeared in one of the two possible realizations, either 


wW, O= X(N — A(t), (4.132) 
or 


W,() = X) — B (t). (4.133) 


If the realization wW is more probable, then it is also more probable that 


the input X(t) consists of the signal A(t) and the interference. In the opposite case it 
is more likely that X(t) contains B(t). Therefore, it becomes necessary to consider 
the probabilities of the different realizations of the interference. 


Let us assume, for simplicity, that the interference is an uncorrelated ran- 
dom function of time. Then, by virtue of the statistical independence of its ordi- 


nates Yp Ype e Yo the n-dimensional probability density of these ordinates will be 
a 
1 l 2 
WY Yo eee Yd) =, "EDH (4.134) 
(y 2r oa) 7 m 


where n can be arbitrarily large. This probability dens ty is the larger, the smaller 
the sum of the squares of the ordinates, or, in other words, the smaller the mean 
square of the interference the more probable it becomes. T ierefore if the inequality 


T T 
t » 
-FJ (Xara + f x—By it, (4.135) 
“u v 
holds, it is more probable that the signal A(t) i the one transmitted. If 


fo- Aydt > + 4 fae B} dt, (4.136) 


then it is more probable that the input X(t) corresponds to the signal B(t). By defi- 
nition the probability of error in an ideal receiver must »e minimal. Therefore, the 
closeness of two functions must be estimated in the iteal receiver by the mean 
square of their difference. 


Let us consider the condition under which the ideal receiver gives a false 
response, Let A(t) be the signal at the input of the c »mmunication channel, i.e., 


X= AO+ X(N). (4,137) 


The response of the receiver to the signal (4.137) depends on which of the two 
inequalities (4.135) and (4.136) holds. This depends ontle form of the realization of 
the interference inthe given case. Suppose that form of the interference is such that 
the inequality (4.136) holds. Then the response ofthe receiver is false. We obtain the 
condition for this error by substituting into (4.136) the trie value of the function X(t), 
given by (4.137), Thus, the condition for the error o’ the ideal receiver is 


T T 
4 f wema> + fiantwo— iea. (4.138) 
0 UV 


After some simple transformations we obtain this condition in its final form: 
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T T 
f woiwo—Aorae > +f Ao@—Bore. (4.139) 
v 0 


The probability of error ofan ideal receiver is equaltothe probability of fulfill- 
ment of inequality (4.139). 


If the method of transmission, i.e, , the functions A(t) and B(t) are given, then 
the right-hand part of (4.139) is some constant function, The left-hand part of (4.139) 
represents an integral transformation of the probability function W(t), From the as- 
sumptions made concerning W(t) we obtain that the left-hand part is a normal ran- 
dom variable whose mean square, according to (3.132), is 


T 
=s f 14 () — Bt)? dt. (4.140) 
o 
Therefore, the probability of error of an ideal receiver becomes 
œ i _# 
Per =: [= dz, 4.1 
F VA z (4.141) 
where 
T 
M=} [a0 Bta. (4.142) 
o 


We shall introduce into (4.141) a new variable of integration 





= 2 
2S 5% (4.143) 
Then we obtain: 
o. a 
ae f i? dx=W(N 4.144 
Per vg ‘ (N), (4.144) 


where 


N=zyV f4o- sora. (4.145) 


The value of the function Y° (N) is easily found from tables of the Gaussian 
error integral (x). Some values of (N) are given in Table 3; Table 3 





YN) | 0.5000 | 0.3085 | 0.1587 0.00135 | 3-107 


The calculation of the probability oferrorisverysimple. Knowing the method of 
transmission (the functions A(t) and B(t)) and the intensity of the interference (the quan- 
tity S)), integral (4.145) can be calculated, and the probability of error of the 
ideal receiver can then be read off Table 3. 


The probability of error of an ideal receiver is a characteristic of the employed 
method of transmission. It shows how far, for a given transmission method, the ideal 
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receiver can withstand interference, or, in other words, it characterizes 

the potential noise- stability of this method of transmission. Ina real receiver this 
stability is lower than the potential one. The difference in noise-stability of an ideal 
and a real receiver is a measure of the perfection of the latter, 


Let us apply the obtained results to telegraphy, emp:oying pulses of both signa 
of amplitude U and duration T. The quantity N in this case is equal to 


N=UYy y: (4.146) 


To increase the noise-stability larger values of N should be sought. By for- 
mula (4.146), this can be obtained in our case by increasing the amplitude U and the 
duration T and by decreasing the interference intensity i. It should be noted that 
for S— œo we have N—=>0, i.e., Perr—>0.5. Thus, at extremely strong interference 
the receiver responds correctly and falsely with equal provability. This means 
that one could as well dispense with calculations of the received signal X(t) and de- 
cide on which signal of the two has been transmitted by tossing a coin. 
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Chapter Five 


RANDOM FORCE ON A NONLINEAR SYSTEM 


§ 26. Simplest Problem of Random Force on an Inertialess Nonlinear System 


The simplest problem of a random force on a nonlinear system, which is 
the subject of the present section, can be formulated as follows. Let a ran- 
dom function x = x(t), describing the external force on the system, be given. 
As was shown in the second chapter, this means that for any n, an n-dimensional 
probability density wie, Kore X n? is known, where the x p žge oX are the va- 


lues of the random function at ine times tp t pov k 


Furthermore, the determinate function y = f(x)is known, which characterizes the 
nonlinear system under random force. The variable y is the response of the 
system to the external force at some instant t; it is determined only by the magni- 
tude of this force at the same instant and is independent of the previous course of 
the process, This property of the system is characterized by the term" inertialess"'. 


The problem consists in finding under given conditions tne n-dimensional pro- 
bability density wlyy, Yor PA for the values MELEE A of the responses of the 


system at the times t fot 


r t- a 


Let us find the probability for the simultaneous fulfillment of the following in- 
equalities: 


yi — 40 LIL <yits l dy, 


i 1 
HFS ASHAT dy,. 
En REER Tile eais (5.1) 


1 
Yn TnS sey delays 


* ë * 
This probability is equal to wly,, Yoo: Ya dy; dy, <e dy Therefore, if 
the joint probability of fulfillment of the inequalities (5.1) is known, we ob- 
ro ë x * 
tain immediately the required probability density wy), Yor -> DAR 


Let us characterize the set of n numbers Xp Xy ea by a point of n- 


2’ 


dimensional space with the coordinates Xp x oX We shall call it the repre- 


2° 
senting point. Then it can be said thatthe responsesy,, Yor +++s¥, are functions 


of the position of the representing point in the n-dimensional space. 


All inequalities (5.1) are satisfied if the representing point is contained in a 
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certain region D of the n-dimensional space. The probabi. ity for the fulfillment of 
these inequalities is equal to the probability for the representing point to be in the 
region D. Therefore the required probability can be writt:n as 


WON. Yp eee May dy, ... dy, = 
n 
== JTT f wlan xa es By dx dy oo. dey (5.2) 
D 


with the integral taken over the region D. 


The right-hand side of (5.2) can be calculated without knowing the boundary of 
the region D by the following device. We introduce under the integral the factor 
A (xp 5 ee x,) which is equal to unity if the represent:ng point is in the region D, 


and equal to zero if it lies outside this region. Then integra-ion over the region D can 


be replaced by integration over the entire n-dimensional space and equation (5. 2) 
can be written as follows: 


WEY, Yp ee J dy, dy, .-- dy, = 
= aei My Xe o o Np) WK Xas -o Mp dg. dy (5.3) 
-0 


The factor A can be explicitly expressed by means of the Dirichlet integral 





+o 
=} f snaz one dz, (5.4) 
z 2 
a 


which is equal to unity if2< y < a,and equal to zero if yis outside this interval, 
We shall set in (5.4) 


1 
a = gA (5.5) 
and P . 
T=I,— MN ES (5.6) 
Then we obtain from (5.4) 
co 1 
oa Polke) ong 
see S z exp[jz, (1 (,) — y} dz 6.7) 
Since 
sin (5 dy, - a)j=5 dy, Zu (5.8) 
equation (5.7) takes the form 
d ° 
i= Zt f exp [/z,f(«,)] exP(— Jz, 2i] 42,- (5.9) 


In view of (5.5) and (5.6) we conclude that the value of òi is equal to unity if 
Yi is within the limits defined by the first of the inequalities (5.1) and vanishes 
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if yy is outside these limits. Analogously, it is not difficult to obtain expres- 
sions similar to (5.9) forthe responsesyo, Yas eeeh: The value of the factor A 


becomes — A(x, Xpo., X,) = 38... 3,5 
+æ 


d ae. aly, w 
=A f exp pias Hasa +o 
cee 2,5 (*n)}| : exp[— i (yiz +yz H+ Yažn)) dz dz... dz, (5.10) 


Substituting this result in (5.3) and omitting the no longer necessary asterisks 
from the responses yp y5 aa yt, we obtain: 
+00 
| — fa 
WY Yar eer In) = Tay Beaters (2, Zen eves Za) X 
-0 
X expl—j OZ + Y2 +- H Ynn) dz, dz... dp, (5.11) 


where 
+00 


Alii Zap vee an) = [Tf emp (2S + af le) + 


wee EZS (Ky) ) W(X Xr oe, Mp) AH, Ay... AX q. (5.12) 
From equation (5.11) follows that the function A(z, Zo: see) is the char- 
acteristic function of the n-dimensional probability density wly,, Yoo ves BARE 
+0 
pea 
A(z; Zar esas z= f ees fon an eee Yn) X 
-0% 
X expli Yi + Zey: + --- F 2nYa)l dY, dye... An. (5.13) 


The obtained relations (5.11) and (5.12) determine the n-dimensional probabil- 
ity density wly,, Yor ++: Yo for the general case, and they represent the required 


solution of the problem. From these relations the particular but important solutions 
for n = 1 and n = 2 result automatically. For n= 1 we have: 


w(y) = x f A(z)exp |[— jyz] dz, (5.14) 
where 
A(z) = f exp Lzf (x) w(x) dx. (5.15) 
For n = 2 we have 
+o 
win wage f f Aee ewt—Jner+ryeedldedz, (516) 
-o 
where 
+o 
A(z. z= f f expt (2,F (41) + ZS (X1 W(X, x) dx, dx. (5.17) 
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The direct application of the general expressions (5 11) and (5.12) or of their 
particular cases (5.14) and (5.15) for n = 1 and (5,16) and /5.17) for n = 2, leads 
sometimes to cumbersome calculations. If to the functior f(x) corresponds a single- 
valued inverse function x = (y) of a sufficiently simple structure, then the following 
device is more expedient. We replace in (5.12) the integration variables Xp Xy e ok 


by the new integration variables Yp Yp oyy Then we have: 


+o 


A C2 tay ooo a) f TT f xP ey t+ zaya + + 2X 


KX we CMs PD -e PODPO). 
-- E (Yn) dy, dye... Age (5,18) 


Combining the equations (5.13) and (5.18) we obti.in the following simple 
result: 


WY Yar eee Mad = 

= w lp (YO. PID +6 PID P ID P Y- Mn) (5.19) 
where w.{9(y,), P(Y.) ..., F(Va)} is the n-dimensional probability density for the 
random variables Xj Xy eee KX, under the substitution x, =9(y;). X, VY), eee 


x, =q(y_). For the particular cases (5,15) and (5.17) we obtain, analogously, 


n 
forn=1 
w(y)= wy le (yl p (y) (5.20) 
and for n = 2 
wY Y) = wle) PONA OY P Wa). (5.21) 


It should be noted that when the domain of existence of y does not extend from 
-o0 to +00 but is smaller, the integration of (5. 13) and (5. 18) has to be carried out 
over this domain, but the validity of the equations (5.:9), (5.20) and (5,21) is 
not affected. 


In many cases the exhaustive characterization of th» statistical properties of 
the response y, as given by the n-dimensional probability density wly,, Yor eee Yo 


is superfluous, and it is sufficient to calculate the first few moments of the response. 
To find these moments we proceed as follows. In the expression (5.15) for the one- 
dimensional characteristic function, we expand the exponential factor of the inte- 
grand in a power series; 


exp Liza = 1+ 2A | 4 Uefa (5.22) 


Then the function A(z) can be written as the following power series: 


a= ¥ Suz), (5.23) 
PE 
where 

m, = f (f(x) w(x) dx. (5.24) 
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Relation (5.23) is the well-known expression of the characteristic function in 
terms of an infinite set of moments. Therefore, relation (5.24) permits to cal- 
culate one-dimensional moments of any order forthe responsey. Analogously, by 
setting in (5.17) 


exp ls {aS e+ zA (42) = 
= re Seas (oy) 
pz v20 % 


-=X (Ae tt GaP Ga (5.25) 


we reduce (5.17) to the following forms, 


+» 
Ae = YS 
iad 
where the moments of the two-dimensional distribution of the response y are 
given by: 


a U” Ven". (5.26) 


mpe = f J UC (say wl, 1dr, dx, (5.27) 


Let us note the important particular case of expression (5.27) corresponding 
to w= v= 1; 


+o 
m, = f f S (4) f (42) w(x, £2) dx, dx (5.28) 


Similarly it is not difficult to obtain from the expression (5.12) the general re- 
sult for a moment of any order of the n-dimensional distribution of the response. 


In conclusion let us note a special feature which is typical of nonlinear trans- 
formations of random functions, Whereas, in a linear system, any moment of 
response is determined by the corresponding moment of the input of the same order, 
in a nonlinear system for the calculation of a moment of response it is necessary to 
know the corresponding probability density of the input, i.e., the infinite number 
of the corresponding moments. 


§ 27. The General Problem of Random Force on an Inertialess Nonlinear System 


The method of analysis which was presented in the foregoing section, can 
be also applied to the following much more general problem, Let k random inputs 
be applied at k points of an inertialess nonlinear system. We have to find the statis- 
tical properties of the m responses of the system at its various points, responses 
yj being related to the external forces by the equalities 


NHN r Xr -e See 
Ya = fr (Xi Xg -e Xr), 


eo 6 e eo sooo ‘l o 


(5.29) 


Ym = fmhI Xz oe oe Xn): 
Each of the inputs xi is characterized by an n-dimensional probability den- 


sity. Since in the general case all the inputs are statistically interdependent, their 
joint k-n - dimensional probability density must be given. 
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For each response of the system an n-dimensional probability density must 
be determined. In view of the statistical dependence between the responses, an 


m-n-dimensional joint probability density must be found for them. 


The solution of the above problem does not present any fundamental difficul- 
ties. But, in order to avoid cumbersome calculations, we shall solve it below for the 
particular case ofk = m =n = 2, The generalization of -he results for any k, m 
and n is obvious. 


With the above limitations, the system of equations (5.29) reduces to 


Jr =E fi (Xi 4). 
Y= f(X. x3). (5.30) 


For the external forces the four-dimensional probebility density 
w(x Xiz Xs; Xoo) is given, where the first index gives the ordinal number of 


the input, and the second shows to which of the time instants, ty or to the in- 


put corresponds. It is required to find the four-dimensional probability density 
ws. Yio Yop Yoo) where the indexes have the same meaning as above, 


Following the method of the preceding section, let us find the probability 
for the simultaneous fulfillment of the following four inequalities, 


. 1 . 1 
Yn F Vu S Yn SIn tg Wir 


. | . 1 
VaT TV SIS Itz Wa 
arn ley ae (5.31) 
In — J Pa S In SIn + 7 Vn 
3: 


A 


. 1 . 1 
Yn F nS Yn <n trn 


* * * * 
which equals wY Yy You Yoo) FY, 199 99% 1%: 


The values x v and x„„ of the external forces will be considered 


iv *12" *2 22 
as coordinates of the representing point in four-dimensional space. By analogy with 
(5.2) we have for the four-dimensional probability density of responses: 


w (Yw Vie Ya Ya) Yu Yia War Wg = 
=f f ffoes Xi» Xar Xn) dX dX, dX dXz (5.32) 
D 


In order to extend the integration of (5.32) over the əntire space we introduce 
into the integrand the factor 


A = b, e 8g e 8s o Ba (5.33) 
In order to obtain the co-factor a we set in the Dirichlet integral (5.4): 
1 (5.34) 
a=a dy 
and 
T= Yna =S (Aw Xn) — Sv (5.35) 


Then we obtain 


+a 


3, = Bn f exp [jz f (Fie Faen [ Jaa] dze (5.36) 


-x 


The co-factors ñy 8, and ô, have analogous forms. Therefore, the expres- 


sion (5.33) becomes 
S (Xir Xiz Xa Xa) = 


=u dot ie wef ff feo EAA (fun Xn + 


+ zafi (X12 EEES IN Xa) H Zf Xa) X 
x exp [— j (Yi + Yia t+ Yaz t Yu?) dz, dz,dz,dz,. (5.37) 


Introducing the factor A into the integrand function (5.32) and omitting the 
asterisks from the responses y, we obtain: 


Wir» Yiz Yz S22) = 


raf ff fae Zz Zs, zD exp l— J (yz. + 


F Yiz + Ya? + Y222)| dz, dz:dz; dz, (5.38) 


where 
A (Zi Za 23, 2) = 


=S f f SEA En, Xa F afi (Lie, X22) F 
F afal X21) + Ufo (Xie Xn) X 


X W (Xir Xiz: Kars X22) AX yy AX yy dX yy dX y2. (5.39) 


Here the function A(z), 24) is the characteristic function of the four- 


): 


Zo Zg 
dimensional probability density wY Yir Yar Yoo 
+œ 
A (Zi Zz 23, zd=f fS f fow Yiz Yar Yoo) EXP IJ (Zyn + 
-w 


F Yia F Za Var + 2 N22) dY dY i2 dYa dYa: (5.40) 


If to the system of functions (5.30) corresponds a sufficiently simple system 
of inverse functions 


Xi = P (Yi Ya. ) (5.41) 
X2 = P(Y Yz} 


then the probability density wY ) can be found more simply in the 


Yir Yor Yaa 
following way. Let us replace in equation (5,39) the integration variables 
Xp Xo Xap and Xoo by new variables Yip Yio You and Yag Then, in accord- 


ance with the rule for changing integration variables in multiple integrals, we obtain: 
+o 
A(Zy, Za Z» 2) = SSS EUC taya 2Ya t+ ZY X 


-%0 
X DIPY Yad Pir Iad Pi Ya) Pe Yau YAX 
X | D,D | dYn dys dYa dyaz (5.42) 


89 


where D, and D 2 are the functional determinants 
Deyn Id, D Ou Ja) 
Ia 


Ov. (Yi Ya) aliy Yn) | * (5.43) 
Yu Ya 


d, Gir Yn) Qı Go Ya.) 


OF k fa Yn) d ye Yn) 
OYn 


Combining the expressions (5.40) and (5.42) we obtain the final result: 
wir Yir Ya Yer) = w31 Wir Yn) Pı Wir Yn). 
P2 Wir Ya) Pr Nias Yal | D2". (5.45) 


D, = 


D,= (5.44) 


where wy is the four-dimensional probability density o: the inputs, with the 
variables changed as indicated in (5.45). 


The transition from (5,39) to(5.42)and, consequently, the expression (5.45) 
are correct if the following conditions are complied with: 1) the functions ti and 


Pa and their partial derivatives are continuous in the domain of integration; 2) the 


product of the determinants D,D, does not change sign in this domain; 3) there is 

a one-to-one correspondence between responses andextemialforces. The last condition 
limits the range of application of formula (5.45) and of its possible generalizations 

to cases in which the number of the inputs of the system is equal to the number of 

its outputs (k = m). Formulas obtained by generalizatior of (5.38) and (5.39) are 
free from this limitation. 


To obtain the relations for the moments of the r:sponse we write the 
exponential factor of the integrand of (5.39) as a power series: 


exp ls (241 (Si Fas) + fi (Xi X22) + 
F 23f2 (X11) Xa) + S(t tn) = 


= > (Aa Cerne tl" AEN E LENE M AELE (5.46) 


valralvgtyal 
wey A 
Then (5.39) becomes: X Ge)" UZ U2 z)". 
AlZ Zn 23, 2) = 


= D MASE Gen" URDU", (5.47 


Yao Yes Yh w 
where the four-dimensional moments ofthe responsesy, and y, are defined by 


Metante = 


+o 
= S S S Pier th Gun eee dX 


X Ua (Sree Lead)" (Xi Xir Ins Xa) dX, dX, AX q, dX qq. (5.48) 


For the mixed moment of the fourth order we have from (5.48), for 
y= y= y= n=l: 


+0 
m=f ff fr (Xi X21) Si is Sed fa (Xr X1) Se (Xi Xn) X 


MK WK, Xiz Xas Xn) dX, dX pq dXq, dX qq. (5.49) 
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Our problem has thus been solved completely in principle. It should 
be borne in mind, however, that the application of the obtained general relationships 
to specific cases meets sometimes with considerable computational difficulties, 


§28. Random Processes in Inertial Nonlinear Systems 


The results of the two preceding sections lead to the conclusion that the 
solution of the problem of random processes in inertialess, nonlinear systems 
does not present, in principle, any difficulties. But the calculation of the 
moments ofthe responseis much more complicated in such a system than in an in- 
ertial linear one, as much more subtle features of the input have to be taken 
into account. 


The difficulties increase sharply if the nonlinear system possesses inertia. 
The reason for this can be readily grasped by taking into account that the behavior 
of such a system is described by a stochastic differential equation, or by a system 
of such equations, and considering further that at present there are no general 
methods of solution of nonlinear differential equations, whether for random func- 
tions or for determinate functions. Owing to this, the theory of random pro- 
cesses in inertial nonlinear systems is at present still in its initial stages. 


The most general problem concerning a random process in an inertial nonlin- 
ear system with one input and one output (for definiteness we shall consider below 
only such systems) is the following. For any n, the n-dimensional probability den- 
sities of appliedinput are given: it is required to find the corresponding multi-dimen- 
sional probability densities for the responses of the system. 


Such a problem is already difficult for an inertial linear system (with the ex- 
ception of trivial cases, when the input is normally distributed, or when there isno 
statistical dependence between values arbitrarily near in time), Far from being 
complete, the results hitherto obtained are of only preliminary character (see for 
instance, /25/), For inertial nonlinear systems even such results are lacking. 


The problem of random processes in nonlinear inertial systems is substan- 
tially simplified if the external force is uncorrelated. In such a case the Fokker- 
Planck equation can be used for its solution. 


The Fokker-Planck equation was obtained while developing the theory of the 
Brownian movement. A rigorous derivation of this equation is given in the work of 
A.N. Kolmogorov / 26/ , a simplified derivation is contained in the book of 
M.A. Leontovich /27/, in which it is called the Einstein-Fokker equation. The 
method of applying the Fokker-Planck equation to the analysis of random processes 
in nonlinear systems is shown in the work of L. Pontryagin, A. Andronov and 
A. Vitt /28/. 


By using the Fokker-Planck equation a number of investigators obtained im- 
portant results concerning random excitation of tube oscillators. The first results in 
this direction were obtained by I.L. Berstein /29, 30/. Along with the works of 
Berstein, the works of G.S. Gorelik /31/ and S.M. Rytov /32/ on the same sub- 
ject should be mentioned, as well as the article by P.I. Kuznetsov, R. L. Stratonovich 
and V.I. Tikhonov /33/. The same problem was solved in a different way by 
I.S. Gonorovski /34/. 


Without belittling the great importance of the results obtained by the above 
authors, it should nevertheless be borne in mind that the case of a nonlinear system 
under uncorrelated action is far from exhausting all the important physical and 
technical problems relating to random force on nonlinear systems. Thus, a 
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further search for methods of analysis of random processes in nonlinear inertial 
aystems is necessary. 


The simplicity of the laws for the transformation of moments suggests that 
the search for laws of response distribution be abandoned and that we should 
be content with the calculation of its moments which, in many practical cases, give a 
sufficiently complete idea of the random process. But this simplified formulation 
of the problem, whitch leads to simple solutions in linear problems, turns out to be 
rather cumbersome for nonlinear systems, This is due to the fact that, in contrast 
to linear systems, where any moment of the response is determined by a moment 
of input of the same order, in nonlinear systems any moment of the response 
is determined by an infinite number of moments of the inpu:, as was noted at the 
end of §26 with respect to inertialess systems, Therefore the expression for any 
moment must have the form of an infinite function series. Similar results were 
obtained by V.S. Pugachev /8/, as well as by P.I. Kuznetsov, R.L. Stratonovich 
and V.I. Tikhonov /35/. 


While the methods of solution of the problem of calculating the moments of 
the responseof the system as given by the above-mentioned investigators give, in 
principle, accurate solutions,they are very cumbersome, which greatly reduces 
the possibility of their practical application. 


It follows from the above, that, when solving such problms one often has to aban- 
don the use of some general methods of analysis, and one has to solve each 
problem approximately, by applying special methods, adapted to the peculiar fea- 
tures of the problem. As examples of such an approach to this question we mention 
the works by L.S, Gutkin /36/ and V.I. Tikhonov /37, 38/. 


In conclusion, a particular class of inertial nonlinear systems should be 
mentioned, the analysis of the random processes occurr:ng in which made by 
comparatively simple means. These are systems consisting of two mutually 
independent units in cascade connection, the first of which being inertialess and 
nonlinear, and the second inertial and linear. We call a ccnnection "cascade" if 
the output of the first unit is connected to the input of the si:cond. 


In such a system the random function, equivalent t> the input, is subjected 
first to an inertialess nonlinear transformation, and then to a linear inertial 
one. Knowing the distribution law of the input to the first, and using the relation- 
ships of the present chapter, the moments of the response on the output of the first, 
i.e., input of the second unit can be found, and then, by tht methods of the third 
chapter, the moments of the response at the output of the s:cond unit can be calcu- 
lated. 
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Chapter Six 


SOME NONLINEAR PROBLEMS IN THE THEORY OF RANDOM PROCESSES 


§29, Action of Fluctuation Noise on a Detector with Exponential Characteristic. 


In many cases the actual volt-ampere characteristic of a tube diode is well 
approximated by the exponential function 


i= Je, (6.1) 
where i is the anode current of the diode, u the voltage on its anode, Jo the anode 


current at u = 0 and a is some constant parameter. 


Suppose that a diode with the volt- ampere characteristic (6.1) is under the 
action of a fluctuation noise u (Figure 19). It is supposed that the noise voltage is 
stationary and has normal distribution with the mean square o2 » ie. 


wt 
EE 
1 wy 


3 6. 
Yin, e (6.2) 





w(u) = 


Figure 19. Diode under fluctuating voltage 


We assume the normalized autocorrelation function of this voltage to be of 
the form 


ku (5) = e71 COS wot, (6.3) 


Expression (6.3) corresponds to formula (4.31) for the second moment of the 
fluctuation voltage in an oscillation circuit, connected to a source of correlated 
random electromotive force. It is required to examine the statistical proper- 
ties of the current in the circuit of the detector. 


As the process is stationary, the d.c. component of the current in the circuit 


of the detector is the first moment of the current and is independent of time. Set- 
ting in (5.20) v = 1 and taking into consideration (6.1) and (6.2) we have 
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= e — zz bau | du. (6.4) 
= ae wae te] 
From the table of definite integrals / 6/ we take e tollowing formula: 


+œ 


£ fx 
f e- Pr +42 dx = e^? re (6.5) 


+o 
Then, after some simple transformations, we obtain from (6.4) 


l 2 
J= Je? M, (6.6) 


The increase in the d.c. component of the current ur der the action of the 
noise voltage is equal to 


i 2 
U=J.—h= hle Nu i]. (6.7) 


Let us find the second moment of the current. For that purpose we shall 
write down the expression for the two-dimensional probability density of the noise 


voltage: 


2 2 

1 uy — 2k uw, + 4; 

w(u &) = ————— oo Soe 

2 
2x3} Vi — 22 203 (1 — +i) 

wherek = ke) is the normalized autocorrelation functicn of the voltage acting on 

the detector. Now, in view of the formula (5. 24), we can write 
1 


morn S fieren eg x 


x exp [= ae) (ui — 2k, a0, + “| du, du, = 


(6.8) 





aE fm [- er +o] Ma 
x Í "w| tees +t e a] a, (6.9) 


The calculation of the inner and the outer integrals in this expression is easily 
done by applying (6.5), and gives the following simple resul:: 


mi? (x) = exp [aah (1 + koj= Bee Os, (6.10) 
At t = 0 we have ka = 1, and we obtain for the mean square of the current 


the expression š 


2 
Sie’ *w, (6.11) 
The mean square of the noise component of the currert in the circuit of the 
detector is determined as follows: 


3 
goad Aaa (4-3), ean 
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Let us examine the expression (6,10) for the second moment of the current. 
Replacing the exponential factor of this expression by a power series and taking in- 
to consideration (6.3) we obtain 


maar See ea pial Seu , 
n=0 (6.13) 

By expressing the factor cos” w? in each term of the series (6.13) as a sum 
of a constant number (for even n) and cosines of multiples of the angle, we find 
that the moment mi!" (3) contains an aperiodic component, which decreases with in- 
creasing t, and damped oscillation components of frequencies wo, 2u9, 3u, ... 
From the point of view of the detection process, the aperiodic component, related 
to the terms of the series corresponding to even values of n, is of fundamental 
interest, 

Let us isolate this component. As well known, the constant term for 


cope is equal to (2m)! /2?™m!)?, Therefore, the above-mentioned component 
is obtained from (6.13) in the following form: 


(a? ze -312m 
mirer (95 Ser So ; (6.14) 


Taking into consideration the well-known relation from the theory of cylindri- 
cal functions 


no Èli”. (6.15) 


where /,(z) is a modified Bessel function of zero order, we can write (6.14) ina 


more compact form: 
MS er (1) = É + My (ase). (6.16) 


We shall find the spectral density of the current, which corresponds to the 
aperiodic component of the moment in the following way. Applying the formula 
(3.62) to individual terms of the series (6.14) we obtain 


2a, -ut 
2 ifi Ot coset drt = 
(6.17) 


2 B (ao,)'” 2ma 
Peia Tear Er 


Therefore, the complete spectral density, as determined by the aperiodic 
component of the moment, becomes 


sm 2m 
Fi aper (w) = EŞ oy urn (6.18) 
mat 


It follows from this expression that the considered component of the spectrum 
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of the current in the detector circuit has a maximum atw==0, and decreases mono- 
tonically withincreasingfrequency. It can be called low-frequency component. It 
appears as a result of rectified noise voltage. 


Analogously, we can single out from the series (6.13. damped oscillation com- 
ponents of the spectral density with maxima in the ranges of wo, 2wo, 3wo, ... 


respectively. We note that the spectrum of the input voltege has its only maximum 
in the frequency range mp. 


The appearance of maxima of the spectral densit’ of the response for 
harmonics of the frequency of the input spectrum maxinum is characteristic 
of random processes in nonlinear systems. The aperiodic component of the 
response spectrum can be missing in certain cases; this occurs if the character- 
istic of the nonlinear element is an odd function. 


$30. Statistical Properties of the Noise Voltage Envelope 
at the Output of a Selective System 


It has been shown in § 21 that the noise voltage at the output of a selective 
system, tuned to the frequency w, is an oscillation of frequency wọ, modulated at 


random in frequency and phase. In some cases, and particularly in analyzing the 
action of such a voltage on the amplitude detector of a radio receiver, it is impor- 
tant to know the statistical properties of the random amplitude of this voltage. This 
question was considered in the publications of V.I. Bunimcvich /4/ and S. Rice /39/. 
We give in the following an account of the results stated in these works. 


The instantaneous noise voltage on the output of a linear system is expressed 
by formula (3.26), in which, by limiting ourselves to the consideration of a station- 
ary random process, we assume the lower integration limit to be equal to —.v, 
and the upper limit to be ty = 0, This last assumption can always be made by choos- 


ing a suitable time origin. Let us denote the integration variable by t’. Then 


0 


wou (O= fin VEC Ed’. (6.19) 
-2 
By changing the integration variable:t=-t’, we obtair 


tou (O) = f win (EM dt. (6.20) 
o 
Let us analyze a single oscillation circuit, conne: :ted to a source of 
noise electromotive force. More general results can be obtained analogously. 
We shal) consider them later on. 


The impulse characteristic of a series oscillation cir cuit can be expressed as 
follows: 


w? 
EO = e sin wt wage sin wef, (6.21) 


with notations as in $ 21. This result can be obtained in various ways. In particular, 
if the transfer ratio operator of the circuit is obtained ‘rom (4.22) by the substi- 
tution j/w=p, and considering that the transform of the unit impulse function 

is equaltounity, then the inverse transformation of the produc t of the above -mentioned 
transforms leads to (6. 21). 
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Substituting (6.21) in (6.20) we obtain; 


œ 
uou (0) = f uin (P) mge -* sin wt dt (6.22) 
o 
As had been shown already in § 21, the noise voltage at the output of a selec- 
tive system is the result of superposition of infinitely many damped elementary 
oscillations, which are caused by elementary impulses of the input noise. A single 
elementary oscillation 


Al on = Uin (t) dt - wye—** sin wot (6.23) 


can be represented by a vector of length u,, (t)dt-wye-** and of phase w, 


(Figure 20). Expression (6.22) gives the projection of the resultant vector, repre- 
senting the total oscillation, on the abscissa. As it is the length of the vector which 
is being considered, one has to write down the expression for its projection on the 
ordinate. This is equal to 


ao 
Vou (0) = f Win (t) we- cos wt dt. (6.24) 
o 
The projections Vout and Voit are random variables. If the input noise is un- 


correlated, as will be assumed in the following, these projections are distributed 
normally. The normal law applies also to correlated external force having a nor- 


mal distribution. The mean square of the projection Usut can be easily computed, 


using equation (3.132). Assuming in itt == 0, and noting that a < wo, we have: 


oo 
2 
c= S flne- sinagi dia 1° s, (6.25) 
o 


Mon 
iS 
ke— tour 


Figure 20. Vector of the elementary noise voltage at the 
output of a selective system 


The same result is valid for the mean square of the projection of v Con- 


out’ 
sidering the equations (6,22) and (6.24) as a set of two integral transformations of 
the random function, it is easy to calculate the mixed moment of the projections u 
and v. For this we make use of equation (3.153), in which we put to = 0, denote 


tc to = ty =t and change the sign of the integration variable. Then this equa- 


tion assumes the form: 


mY (2) = sf By (t+ 1) 8a (6) dt. (6.26) 
0 
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In the present case of t = 0 the impulse characterist c $, (f) is determined by 
equation (6.21), and &(t) is 


£3 (f) = we"! cos wgl. (6.27) 


In view of the above, we have: 


mie (0) = S f age-*sin wyt) lage" cos wt) dt = 
ô 


= $ Set f e-^ sin Qagf dt = =7Sig ; (6.28) 
0 


From (6.25) and (6.28) we obtain the correlation coefficient between u and v: 


mi9 (0) a 
Pue (0) = >r = THe’ (6.29) 
Since a < wy we can assume 
Pw (0) = 0, (6.30) 


in other words, the projections can be considered as statistically independent of 
each other. 


After this preliminary work we can proceed to the study of the distribution law 
of the amplitude U of the fluctuation oscillation, which is connected with the projec- 
tions u and v by the nonlinear relationship 


=Vet+e. (6.31) 


For the solution of this problem we shall use the relations which were obtained 
in $ 27 foran inertialess system with several inputs and outputs. The present case 
can be considered as having two inputs and one output, and it is required to find the 
one-dimensional distribution of the response. Equations (5.38) and (5.39) reduce 
to 


+0 
wow =H f A(z) eM" dz, (6.32) 
with pi 
+o 
A(z) = J J exp 12 feu Al Oy Xa) džu dän. (6.33) 


In our case we have X14 7 Ws XQy 7) yF U. The two-dimensional joint 


probability density for the projections u and v will be 


p -22e 
w(u, 0) = œ (u) w (0) = yat w. (6.34) 


Substituting (6.31) and (6.34) in (6.33) we obtain: 


A= se ffe SF AVI du do. (6.35) 


Here the integration is performed over the entire uv-plane. We note that 


the integrand of (6.35) is constant for U == V u? -+ of = const. This allows to replace 
integration with respect to the coordinates u and v by radial integration, i.e., sum- 
mation over annular bands of radius U and width dU, centered atthe origin. Because of 
the constancy of the integrand, the integral with respect to a single ring is equal to 


g 
QnU dUe e”, (6.36) 


In view of (6.36) we can express the characteristic function A(z) as follows: 
ry D 
aw= f ae eta. (6.37) 
o 


Combining (6.37) with the definition of the characteristic function 


A(z) = f wax (6.38) 


and considering that the amplitude U can have only positive values, we conclude 
that the amplitude probability density is equal to 
p’ 
wU)=4e™, (6.39) 


The same result is obtained by substituting the value (6.37) of the character- 
istic function in (6.32) and integrating. But in the present case we succeeded to 
carry out the calculation by a much simpler method. 


The distribution law is expressed by (6.39) is called Rayleigh's law: the 
corresponding relation between the probability density w(U) and the amplitude is 
shown in Figure 21. 


wi} 


0 u 
Figure 21. Probability density according to the 
Rayleigh distribution law 


Let the analyzed noise voltage act on an ideal amplitude detector. We call 
an amplitude detector ideal if its transfer ratio is equal to unity and if it re- 
produces on its output without distortion the envelope of the modulated oscillation, 
acting on its input. The d.c. component of the voltage on the output of the detector, 
which is equal to the mathematical expectation of the amplitude U, is expressed as 
follows: 


œ œo n 
E.=MWI= fueua=s f Ute aU. (6.40) 
0 (1 


In tables of definite integrals /6/ we find the following formula: 


= pat pte gy — Ca — UM T 
fe xa dx = OB Wf 3. (6.41) 


Using a particular case of this formula, with a = 1, we obtain: 


R 
E. = y ze (6.42) 


Let us find the mean square of the amplitude U, whica is at the same time the 
mean square of the voltage at the output of the ideal detector. We have: 


uo œo v 
g= o= f Uw Udu =h fur aU. (6.43) 
0 9 


For the calculation of the integral (6.43) we shall use the following formula 
from the tables /6/: 


w 
, 
f en PH xt dx = So (6.44) 
Qpati 
A pe 
In our case a = 1, and therefore 
o} = 0} = 20, (6.45) 


The mean square of the noise component of the voltage output from the detector 
is determined from the formula (6.42) and (6.45) as follows 


of oh — Et = SSF ot = 0.4308, (6.46) 





Thus, all the more important characteristics of the random amplitude U, 
which could be calculated from its one-dimensional probability density have been 
obtained. Of course, the first and second moment (6.40) ard (6.45) could have been 
found directly, without finding w(U). 


Let us consider now the statistical dependence in the envelope of the noise 
voltage. For this, we shall determine the two-dimensional probability density for 
the amplitudes U and U,, separated by the time interval <. Tne relation between the am- 
plitude U and its projections is shown in (6.31). Analogous.y, we have: 


U,= Vato (6.47) 


As was shown above, the projections u and v in (6.31) are statistically inde- 
pendent. The same applies to the projections uw, and v, Te statistical dependence 


between u and u, is characterized by the moment (4.31), whence we obtain 
Puu, (1) = e~" Cos oyt. (6.48) 


We have to investigate the statistical dependences bet veen the following pairs 
of projections; v and v,, wandv,,vandu,. Applying (6.26) a: d (6.27) we can write for 
the projections vand t, : 


co 
mi" (x) = S f [wen "+ cos wlt H) [mye -t ces ng) dt. (6.49) 
u 
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The calculation of the integral does not present any difficulties. Since a< wo, 
the result can be taken as equal to 


2 


1 
mi") (2) = 7 > Se-* cos wot, (6.50) 
whence we obtain, in view of (6.25) and (6.48): 
m,”? (0) 
Pee, (t) = —— = 6-7 COS Wot = Puy, (1). (6.51) 


Analogously, we have for the projections u and vi 


mi) = S f foe °F sin wg (t+ Alog cos wot) dt = 
o 


2 (6.52) 
= + > Se~ “sin Sot, 
i.e., 
Pur, (1) = @ sin wot, (6.53) 
and for the projections v and u;: 
ao 
mS) (2) = Sf loet coseag(t + t)lluge™™* sin wef] dt = 
o 
2 (6.54) 
1 % - 
=— Gre inot, 
whence 
Prue (1) = — e7" sin wy. (6.55) 


Henceforth it will be convenient to use the folowing notations for the expres- 
sions (6.48), (6.51), (6.53) and (6.55): 


Puu, = Pery = Ẹ (1) COS wot = $ (x) cos B, (6.56) 
Puu = — Pru = P(t) sin wot = (x) sin B, (6.57) 


where ¥(t) is the envelope of the normalized correlation function of the instantane- 
ous noise values, which for a single oscillation circuit is equal to 


po) =e" (6.58) 


Now we can write down the four-dimensional probability density of the pro- 
jections. As well known, with a normal distribution law the n-dimensional proba- 
bility density can be expressed as follows /40/: 

W(X. Xz -os Ky) = 


n 
1 1 Xy 
= exp | — 5r D ’ (6.59) 
043g.+- oq V (22D P | 2D 2, y 9.04 | 


where D is the determinant of n-th order: 








PiP s+ Pin 
D Oe as (6.60) 
Pnn: Pn 


Di, is the algebraic complement of the element Pij of this determinant, Pij being 
the correlation coefficient. For i = j we have Pij = 1. Inthe present case, putting 
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I 7 UW XQ FV, Xg Uy and X45 Vy, we have: 
1 0 cosh ¢snp 
0 1 —ysinB cos p 
D=| cosp —ysinp 1 o | (6.61) 
ysinp Ycosf 0 1 
The calculation of D and of its algebraic complements Dij gives the following 
results: 
D=(1— p}, (6.62) 
Dn = Dz = Dy = Du = I— p, (6.63) 
Da = Dy = Dy, = Dyu = 9, (6.64) 
Dy, = Dy, = Dy = Dy = — 4 (1 — p) cosp, (6.65) 
D,, = Da = — Oy = — Dy = —ẹ (1 — 4?) sin p. (6.66) 


With these results the general expression (6.59) becomes: 
1 1 
w(u. v, u, U,) = moal- ai y fuz + o+ 
+ iit vi — 2y icosp (uu, + vv) + sin p (se, — vu). 


The characteristic function of the probability density of the amplitudes U and 
U, can be expressed thus: 


Alz z= f f f J ewl la VEFE ta EF AX 


Xw(u, v, ü, V) du dt du, dv,. (6.68) 


(6.67) 


by analogy with expression (5.35). 


Let us introduce new integration variables, which are connected with those of 
(6.68) by the relations 


u = U sinb, v= Ucosé, u, =U,sin6,, v, =U. cos 9,. (6.69) 


Then, using the rule for change of variables in multiple integrals, we give to 
the characteristic function Be 68) the form 


A(2,, 2) = [Seu + abana 
Seed aa U+ v3) dU dil, X 
ax an 
xf f exp aay cos (6 — 8, —8)] dô d6,. (6.70) 
v o 


The calculation of the inner integral is simple, in view of formula 


2a 
f ete dx = 2r (o). (6.71) 
o 


This formula is easily obtained by replacing the expone ntial function in the 
integrand by a power series, by integrating by parts and cor paring the obtained re- 
sult with the power series (6.15). Using expression (6.71), v'e obtain: 
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an 2 
Sf exp | sts cos (0 —0,—f)] a a8, = 
0 v 


= rèh wm 5) (6.72) 
which allows to write (6,70) in the form 
Aen 2) = f Í PEU HRUN ayy X 
Xh [item] [- | dU dU.. (6.73) 


Comparing (6.73) with the definition of the characteristic two-dimensional 
distribution function 


AG 2) = f f exp(/(2,U + zU wU, U) dU dU., (6.74) 


we obtain the following final result: 


UU WUU, U? + U? 
20. Uday oats] a 9 
If the time interval rt is very long (t — 0o),then (t) —» 0 and with /,(0) = I we have 
m Ur 
@(U, U)= A ea. Ue E, (6.76) 


i.e., (6.75) becomes a product of one-dimensional probability densities of the 
kind (6.39). This was to be expected, as at t — cœothe amplitudes U and U, become 
statistically independent of each other. 


The mixed moment of amplitudes U and U, can be found in the following way; 


- mC) =f f UU,w(U, U,) dU dU.. (6.77) 


o o 
where the probability density w(U, me is expressed by formula (6.75). Thus: 


Po=f f ak [ate] x 


U+ U? 


6.78 
xep |- ma awau, Ai 


The calculation of the integral (6.78) leads to rather tedious computa- 
tions. They are contained in the monograph by V.I. Bunimovich /4/ and will not be 
repeated here. We give only the final result: 


P (e) = o? [2E (h) — (1 — 49) KW). (6.79) 


where K(¢) and E(¢Ẹ) are complete elliptic integrals of the first and second class, 
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njà 


S (Geena 1 SERN, 
K = f VaR’ (6.80) 
nfi 
EW =f V1— y} sinto de. (6.81) 
0 


The mixed moment for the fluctuation component of the amplitude and, simul- 
taneously, of the voltage at the output of the ideal detector »ecomes, by virtue of 
(6.42) and (6.79), 


mid (x) = mP N E= e [REH LAE). (6.82) 


As complete elliptic integrals can be expanded into rapidly converging series 
in terms of even powers of y, we can expect that expression (6.82) can be approxi- 
mated with sufficient accuracy by a polynomial with a smal: number of terms. In- 
deed, the expression (6.82) can be very well replaced by the approximate formula 


mP? (x) = 03 {0.3992 (x) + 0.04p¢(x)]. (6.83) 
For tentative calculations the still rougher approximation 
miD (x) = 0.430%? (x). (6.84) 
can be used. 


Results of calculation by the exact formula (6.82) ard by the two approxima- 
tions (6.83) and (6.84) show that the error introduced by the formula (6.83) does not 
exceed 1%, and can reach 10% with the formula (6.84). 


In conclusion, let us examine the spectrum of the envelope or of the voltage at 
the output of an ideal detector. Using the approximate forn.ula (6.84) and assuming 
that the envelope (+) of the normalized correlation function for the instantaneous 
noise values satisfies the relation (6.58), as it does in the case of a single oscilla- 
tion circuit, we obtain from (3.62): 


Tapa: (6.85) 


Fy) = 2 f 0.430te "cos wt dt = 0.550? 
0 


For definiteness, the above analysis has been restricted to noise voltage ona 
single oscillation circuit. But it can be easily seen that thit assumption determines 
only the form of the function $(t). The rest of the obtained results can be applied 
to any selective system. 


In the theory of radio reception, in addition to the abov >-considered problem, 
it is also of interest to examine the statistical properties of the envelope when fluc- 
tuation noise and signal of some shape are simultaneously applied to the input of 
the selective system. In such a case the analysis is accomp ished by the same 
means, but the computation turns out to be considerably more cumbersome, 


$31. Statistical Properties of the Phase of a No se Voltage 
at the Output of a Selective System 


Phase fluctuations of a noise voltage at the output of a selective system are 
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of interest in the case when this noise acts on some phase-sensitive device. The 
following final results have been taken from V.I. Bunimovich /4/. 


We understand by the random phase of the noise voltage the angle formed by 
the vector U, which represents the random process, and the abscissa. This ran- 
dom phase was already introduced in the foregoing section, (the angles 6 and @, 
in the expressions (6,69) and the following). From the first pair of the relations 
(6.69) we find that the nonlinear transformation to be examined is, in the present 
case, of the form 


O=arctg =. (6.86) 


Starting from (6.33) and (6.34) we shall write down the characteristic function 
for the probability density of the phase as follows: 


w+ 


+o 
A(z) = | fer [2 arc tg =| . z e 2” du dv. (6.87) 
-09 


Replacing in (6.87) the integration variables u and v by the new variables U 
and 0, we obtain 


A(z) = f e740. af tg 
v 


A 
2% dU. (6.88) 


Applying formula (6.44) taken from the tables, for a = 0 we can transform 
the equation (6.88) to 
ae 


A(z) =f er I 


. 


(6.89) 
v 


Now, combining (6.89) with the definition (6.38) of the characteristic func- 
tion of the one-dimensional distribution, we obtain 


1 
w (8) = -> (6.90) 
Thus, all values of the phase between zero and 2x7 are equally probable. 


We turn now to the two-dimensional distribution of the phase. The correspond- 
ing characteristic function can be written down, by analogy with the equation 
(5.35), in the following form 


+o 
Alz, d= f S ff ofj {zare E zace x 


X w(u, v, u, V)du dv du, dv, (6.91) 


where the four-dimensional probability density w(u, v u,.v,) is determined by 
formula (6.67). By changing in (6,91) the integration variables in accordance with 
(6.69) we obtain 

A(z, 2)= 


aa te = 
UU. 
= exp [j (2,9 + 2,0,)] d0 d0, Waa X 
J feoves+spnanas, f faite 
U? — X% cos (8 — 0, — P) UU, + U? 
2a? (1 — 4°) 





X exp E ] dU dU,. (6.92) 


105 


In the paper of S. Rice /39/ the following definite integral was computed: 


eo 
ff xy - exp [— (x? + 2xycos p + y?) dx dy == 
o 
= 4 cosec ẹ (1 — pctg q). (6.93) 


If, in the inner integral of (6.92), the variables are caanged 
u ui 


mae mS" a 
and 
pcos (8 — 8, — B) = n= — cos@, (6.95) 


is introduced, then its computation amounts to the use of formula (6.93). In view 
of this the expression (6.92) assumes by simple transformations the following form: 


a= 2 
A(z, = f f expt (2,0 + 2,81 d0 48, 
ow 





1— p} 1 r — arc cos 
x 4x2 { 1— x + n aq— PD a . (6.96) 


Combining (6.96) with (6.74) we obtain the two-dimensional probability density 
of the phase: 


iz 1—ẹ? 1 R— arc cos y 
w (8, Soe (oe tee (6.97) 


where the quantity ņ is determined by equation (6.95). We recall that in the 
considerations of the foregoing section, on which the calculations preceding 
(6.97) were based, it was assumed that f, > f}. In other worcs, the phase § corres- 


ponds to a time moment which follows the phase 6, after an interval t . 


If it is assumed thatt — oo, than $(t)+0, ņn— Qand the probability density (6.97) 
at the limit assumes the value 


w (6, =p: x. (6.98) 


Comparison of (6.98) with the one-dimensional probabil ty density (6.90) shows 
that at t — 00 both phases become statistically independent o) each other, as was to 
be expected. 


Let us consider the statistical properties of the phase 6, when the value 
has been fixed at some time moment, which is being taken as the beginning of the 
reading. For this, we must find for § the conditional probability: 


w (6, è.) 


(8,19) =a: 


(6,99) 


In view of expressions (6.90) and (6.97) for the two probability densities enter- 
ing in (6.99) we obtain: 


(8, |) =A [+a 


= — arc cos y] 


Tee (6.100) 
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As follows from (6,95) the quantity 4 satisfies the inequality O< n< 1. Itis 
easily seen that the probability density w(8,|6) increases monotonically and bound- 
lessly as % approaches unity. 


At fixed interval t the maximum of the quantity and, therefore, of the condi- 
tional probability w/(@,{%) occurs at cos(@—- 0, — p) = 1, which corresponds to the 
condition 


e == 6, +B = 0, + wor. (6.101) 


Expression (6.101) gives the most probable value of the phase 0, if at the 
time moment taken as the beginning of reading the phase were 0, From the right- 
hand side of (6.101) a multiple of 2% was excluded, since, at +==0, it is natural to 
require that $6 =b.. 


As the cosine is an even function, deviation of the phase 8 from its most 
probable value (6.101) causes a decrease of the probability density w(0,{0), which 
is independent of the sign of this deviation. Thus, the conditional distribution of the 
phase § has an axis of symmetry, in a position defined by (6.101). 


As well known, in any distribution which has an axis of symmetry the most 
probable value of the random variable coincides with its mathematical expectation. 
Therefore, expression (6.101) gives at the same time the mathematical expectation 
of the phase at the moment +, whence follows that the vector representing the ran- 
dom process rotates in the mean with the angular velocity w, This conclusion 


agrees fully with the notion of the noise voltage at the output of a selective system 
being an oscillation of frequency wọ with amplitude and phase changing at random. 


We find the height of the maximum of the conditional probability density from 
(6.100) by substituting y= y: 


z — arc cos } 


vr 0 l= H) (6.102) 


At small values of q the envelope $(t) of the correlation function of the in- 
stantaneous noise voltage is near to unity and the probability density max (9, |) 
is large. This means that the possible values of the phase § are mainly concentrat- 
ed near the most probable value (6.101), i.e., the dispersion of the phase is insig- 
nificant. As q increases the height of the maximum drops and the distribution be- 
comes diffuse—the dispersion grows. At t—» oo we have ọ(t)—0 and the distri- 
bution of the phase becomes uniform: 


Wmax (8.19 = w0) = z. (6.103) 


It was assumed above that thc phase J and 0, are random variables with pos- 
sible values between zero and 2r. Inthe expression (6.100) for the conditional proba- 
bility this corresponds to fluctuations of the phase § about its mean value within the 
limits of —tand+ z. Since the selective system is completely indifferent to the 
phases of the oscillations occurring init, the phase can deviate arbitrarily far and with 
any probability from its mathematical expectation. The applied method of analysis 
is such that when the phase passes the limits tnthe terms + x are automatically 
dropped; if this is not done the phase strays from its mean value similarly to the 
Brownian particle considered in $ 19. 
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§32. Statistical Dependence Between Envelopes f Noise Voltages 
at the Outputs of Two Selective Four-poles wit their Inputs 
Connected in Parallel 


The solution of this question is of interest in cases -vhere the outputs of two 
selective four-poles, with their inputs connected in parall2l, are connected to detec- 
tors whose output voltages are added together in some wzy. Such systems are of- 
ten encountered in FM discriminators of radio receivers, which are used for auto- 
matic frequency control of the heterodyne and also for racio reception of frequency- 
modulated oscillations. The rectification of voltages from the outputs of two selec- 
tive four-poles and their subsequent addition is also encountered in diversity recep- 
tion, where, toovercome fading, the reception is simultaneously done by two receiv- 
ers, tuned to different frequencies, whose signals are added together after 
detection. 


In order to avoid cumbersome calculations only a s:mple particular case will 


be considered below, in which the four-poles, tuned to th: frequencies oy and Op 


respectively, have similar transfer characteristics of the form shown in Figure 22. 
We recall that by the transfer characteristic of a selecti 7e system we mean the 
law of steady state of the amplitude of its output voltage upon instantaneously 
switching on a sinusoidal input voltage of unit amplitude. 


The envelope of the impulse characteristic is the time derivative of the trans- 
fer characteristic and has in our case a rectangular form (Figure 23). The high- 
frequency complement of the impulse characteristic has the same frequency as the 
natural frequency of the selective system, as was already shown in $30. The phase 
of this complement is not essential for the following. We can write, therefore, the 
equations for the impulse characteristics of four-poles a: follows; 


Ko inot OKELA, 


= (6.104) 
SO= 9 (t >a), 
sinus (O<t<a), (6.105) 
O= 9 (t > a). 


The noise voltage at the common input of the four- oles will be supposed to 
be uncorrelated. We shall analyze the statistical depender ce between the envelopes 
at one and the same time moment. This allows to compu:e the mean square of the 
noise after addition at the outputs. If the spectrum of th2 resulting noise is also of 
interest, then one has to examine also the statistical dep:ndence between values of 
envelopes separated by some interval t. Such a generalization does not present 
substantial difficulties. 


The projections on the coordinates of the vectors ‘epresenting noise voltages 
at the outputs of the four-poles are by analogy to (6.22) ind (6.24): 


a 
u= fw (2 sin w,f dt, (6,106) 
v 
v= f lin (y cos wt dt, (6.107) 
v 
a 
m= fun () £2 sin wot dt, (6.108) 


o 
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%= f Win (N © cos wy dt. (6.109) 
o 





Eit) 
Uyn (tI Ko 
Soot 7 

' ---—¢ 
; = 
a— a 
Figure 22. Transfer characteristic Figure 23. Impulse characteristic 
of a single selective four- of a single selective four- 


pole pole 


As already established in $30, the random variables u and Vv, can be consid- 


ered as statistically independent. The same applies to the variables u, and v,. 


We calculate the mixed moment for the projections u and uz with the help of 
equation (6.26), assuming in it t = 0 and considering that | w, — w| < o, + wz: 


K? sin boa 
m," = sf snot sin wof dt = 7 oo n (6.110) 
where jw =— w, —w,. If we put dw = O then both four-poles become identical, and 


(6.110) gives the value of the mean square of the noise at the output of each. 





K? 
mi) = o — Jia (6.111) 
The correlation coefficient between the projections u and u, is equal to 
sin Awa 
Puy = pe (6.112) 
Analogously, we find: 
“Ke KÈ sin wa 
a W. 
my ™ = S | z cos wt cos wg df = aa (6.113) 
v 
ie., 
Pou = Puiu, (6.114) 
For the projections u and Vo we have; 
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« 2 | — cosdwag 
min a= f <2 sin w,f COs gl dt me — ae be ' (6.115) 
° 





whence we obtain: 


Pun =— eee f (6.116) 


The expression for the correlation coefficient betwen the projections Uy 


and vi has the form: 


1 — cos doa 
Pus, = — ia (6.117) 


After having found all correlation coefficients between all projections we 
have to make the calculations analogous to those of §30. The determinant D (6.60) 
has in the present case the form 


l 0 sin x 1— cos z 
x x 
0 1 1—cosx sing 
D= x = (6,118) 
sin x 1—cosx i i 
z = 1 0 
"= e 0 1 


where x =Awa. The computation of the determinant (6.11%) gives 


x 


D=[1— 3(1—cos «)/. (6.119) 
The algebraic complements of the determinant (€.118) are 
Du = Dp = Dy = Dy, = 1 — 2,(1 —cos x), ‘eam 
Du = Dn = Dy = eg == 0, (6.121) 
Da = Dy = Du = Da = — [1 — Zt — cos x), roe 
Dy= a = — Da = — Dam — 21-1 — cos a). (6.123) 


Using equalities (6.119) to (6.123), we can write down the four-dimension- 
al probability density of the projections, in accordance wih (6.59) as follows: 


L 
wafi- -5 al — cos a) 


1 
aft Tat. — cos =, [itita (6.124) 


[1 = (uu; + 00) pioa =T Cao — vu)] } } 


The characteristic function of the two-dimensional probability density for the 


amplitudes U and U, is expressed analogously to (6.68). By introducing new vari- 


Whun Vy. Wz, v) = 


xa _ 


ables 
u, =U, sinð,, v,=—U,cos6,, u,—U,sin8,, o, =U, cos 6,, (6.125) 


110 


we obtain A(2,, 2) = SexpU@Ws+2Wd z PeX 
C) atei 
X exp [— zrama (V? + UD] aU. au, x 


ry Uw (6.126) 
x ff exp [atts cos, — 0, — Ba, d6,, d 
do 


where 


— aM _, sin (4wa/2) 
(s xf2 dwa/2 ’ (6.127) 


B = arctg Loes i (6,128) 


The characteristic function (6.126) has a completely analogous form to the 
expression (6.70), This allows to write down without further calculations the two- 
dimensional probability density for the amplitudes U and U, by analogy with 


(6.75): 


UW. WU: UT + U} 
w(U,, U)= pac ee T eens . (6.129) 


As in (6.75), the obtained formula shows that at ọ +0 the two amplitudes be- 
come statistically independent of each other. In the present case, it follows from 
(6.127) that this occurs at Aw = œ or ata = œ, i.e., either for unlimited in- 
crease of the difference between the two frequencies w, and w, orfor unlimited con- 


traction of the frequency bands passed by both four-poles (as known, the pass band 
width of a four-pole is inversely proportional to the settling time a). 

The identity of the expressions (6.75) and (6.129) allows to use for the com- 
putation of the mixed moment of the amplitudes Ui and U, the exact formula (6.79) 


as well as the approximations (6.83) and (6.84). It follows from the latter, from 
(6.46) and from (6.127), that the correlation coefficient between the amplitudes Ui 


and U, is approximately equal to 


a sin (4wa/2)}2 
Pa| euan) ` (6.130) 





Let us examine the change in the correlation coefficient with increased 
detuning between the four-poles. For this purpose we shall examine the graph of 
Figure 24. Itis seen that the correlation between the envelopes of the noise volt- 


ages vanishes practically (f, < 0.05) at dwu > Ža, i.e., at 


a. Af > $ = 0.83. (6.131) 


If it is taken into account that the settling time and the pass band AF of a 
selective system are connected by the approximate relation 


a-AF=Il, (6.132) 
it is easily seen that condition (6.131) is to be spectrally interpreted as a 


detuning in which the two four-poles block non-overlapping bands from the input 
noise spectrum. 


If the two four-poles are terminated by ideal detectors (in the sense of $ 30) 
whose output voltages are connected in opposition (according to the polarity of the 
d.c. components of these voltages) then the mean square of the resulting noise volt- 
age becomes, in accordance with (6.46) and (6,131): 


S: 
o? a= o? + o3 — 2p, ,9,0, = 0.8607) 1 — (6.133) 
> 





Figure 24. Graph of the normalized correla:ion function 
for noise voltages at outputs of four-poles 


This expression shows that the mean square of the resulting noise varies as 
a function of the product Awa between zero and 0.8663. 


$33. Statistical Properties of the Voltage Envelope at the Output 
of a Selective System Under Action of N »nmodulated Signal 
and Fluctuation Noise 


We return to the problem considered in §30 and aiid to it the following condi- 
tlon. We assume that the selective system is acted upon by the fluctuation noise 
as well as by a nonmodulated oscillation of amplitude U, „ and of frequency equal to 


the resonance frequency of the generator, This case was also analyzed in the 
publications cited in the above-mentioned section. 


As far as the instantaneous values of the voltages and currents are concerned 
the system is linear, and, by the principle of superposit:on, the random and the regu- 
lar processes, which take place in the given case, can >e analyzed independently. 
But, as had been already shown in §30, the problem of he envelope is nonlinear, 
and therefore the nonmodulated oscillation signal and tre fluctuation noise must be 
considered together. 


Let us find the one-dimensional distribution of th: envelope of the output volt- 
age. As was shown in $31, all phases of the noise volt.ge vector are equally prob- 
able. In other words, the projection plane uv has complete radial symmetry. 
This allows to rotate arbitrarily the axes of u and v, preserving, of course, their 
mutual perpendicularity. For the present problem it is convenient to orientate the 
u axis along the vector U _ of the signal and the v axis perpendicular to it. Then the 
length of the resulting voltage vector can be written as 
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U =V U, +} P F>. (6.134) 


Applying formula (6.34) for the two-dimensional probability density of the 
projections u and v and the general expression (6.33) for the characteristic 
function, we obtain: 


+00 ne 
A(z) = ek Vtu e | P a du dv. (6.135) 
rot 


We introduce new integration variables U and 8 , connected with u and v by 
the relations 


U+ u =U cosb; v=Usinð. (6.136) 


Then the characteristic function (6.135) becomes 
Awy= fahe = au fer a (6.137) 
= Feat A 


By calculating with the aid of the formula (6.71) the inner integral of (6.137) 
and combining the obtained result with (6.38) we obtain: 


v7.03 
w= $ n (Z) =, (6.138) 





In the absence of a signal (U, = 0) and since I (0) = 1, the expression (6.138) 


transforms, as expected, into (6.39). The other extreme is the case of very strong 
signals. It is clear from geometrical considerations that for very large values of 
Up it is not essential to account for the projection v. It follows, that in this case 


the difference U-U, coincides with the projection u, and therefore 
w-oy 


wUy=-—_ ee” (6.139) 


VIn: 





This result can be also readily obtained from (6.138) by the use of the asym- 
ptotical expression for the Bessel function 


hia) = oe. (6.140) 


Yy aux 


Let us find the mean value of the amplitude U, or, which amounts to the same, 
the d.c. component of the voltage at the output of the ideal detector, connected to 
the output of the selective system: 


2. p? 
U` +U 


En =MU= fuss (e) = au. (6.141) 
v 


For the computation of the integral (6.141) we shall use the formula 
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ao 


Jerene mas = XG am {n (L) +S) EJ) (6-142 


This formula is obtained, if both sides of the Pll) 


J Ig(yxre-Ptdx—=>V = ze 7 Aea (6.143) 
v 


are differentiated with respect to p. 


This is given in the monograph by R.O. Kus'min /41/ which deals with the 
theory of Bessel functions. In view of (6.142) equation (6 141) assumes the follow- 


eey Zet pa (23) 4 [1 (28) 44(28)]) cea 


or, in view of (6.42) we obtain 


2 Sya {i (F)+a[4 ($)+ h (3))}. (6.145) 


wherey = U,V 23:8 the ratio between the effective voltag+s of signal and noise at 





the output of the selective system. 





0 B O 2 2 % 


? 
Figure 25. Dependence of the dimensionless rectified voltage 
on the signal-to-noise ratio at the 
input of the detector 


The graph of the relation E,./E = = f (mn) corresponding to formula (6.145), 


is given in Figure 25. It is evident from the graph that : low signal (y<< l) has no 
substantial influence on the magnitude of the detected vo tage, which is essentially 
determined by the noise. On the other hand, with strong signals (ņn`® 1), the pre- 

sence of noise has little effect. 
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Expression (6.138) makes it also possible to calculate the mean square of the 
resultant amplitude U and the mean square of its fluctuation component. 


§34. Statistical Properties of the Voltage Phase at the Output 
of a Selective System under Action of a Nonmodulated Signal 
and Fluctuation Noise 


In the foregoing section the relations (6.136) introduced the random phase 


Aas the angle between the resultant vector U and the signal vector Uy: This angle 
is expressed by the random projections u and v as follows, 
6 = arctg PEAR (6.146) 
Utu’ i 


Let us examine the statistical properties of the phase ġ. In view of expression 
(6.34) for the two-dimensional probability density of the projections u and v the re- 
lation (6.33) for the characteristic function can be given the following form: 


we 


+00 
aw=f f exp [jz arcte gims] sane œ dudv. (6.147) 





By replacing the integration variables in accordance with (6.136) we obtain: 





v2 
%4" TEE © U2, ct 
A(2)= f æa Saat fue 3 dU. (6.148) 
— v 


For the computation of the inner integral of (6.148) we use the formula 


ER ENE ERTO} (6.149) 
o 


where ®(z) is the error probability integral: 
2 7 -= 
Diz =i f T dx. 6.150 
(2) Ves e ( ) 


Formula (6. 149) can be obtained from the tables /6/ 


fermesa Fi olg). (6.151) 


v 
by differentiating both sides with respect to the parameter q. 


In our case 


p=, gm Uecoss, (6.152) 
Y2e a 


and the application of formula (6.149) gives for the probability density of the phase 
w (6) the following expression: 
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-v 
= Wi 7 + mcos Bertce [1 + Oreos}. (6.153) 


As in the fo>egoing section, we have »=U,/V 2a. 


We note that at y = 0 (no signal) the expression (6,153) transforms into (6.90). 


Figure 26 shows curves of w(6), constructed from (6,153) for different values 
of the ratio 7. It can be seen from this graph that the increase of the signal amp- 
litude results in a concentration of the most probable values of the phase around 9 = 0. 
This means, that under the given conditions the phase of ihe resulting oscillation is 
to an ever increasing degree determined by the phase of tie signal. 





Figure 26. Graphs of the probability density of the phase for 
different signal-to-noise ratios 


$ 35. Computation of the Detector Output for Given Statistical Properties 
of the Applied Voltage Envelope 


It was shown above that the examination of the statistical properties of the 
voltage envelope at the output of the selective system gav» at the same time equiva- 
lent results concerning the statistical structure of the vol-age at the output of an 
ideal detector. If the distribution law of the envelope is known, it is easy to cal- 
culate also the detection effect for the most varied for:ns of the volt-ampere 
characteristics of an inertialess detector. For this we heve to know the relation 
between the mean current value in the detector circuit and the amplitude of the ap- 
plied alternating voltage. We call this relation the detec or characteristic. 


We shall consider first a detector with an exponential volt-ampere character- 
istic, as expressed by the equation (6,1). If a harmonic voltage 


u = U cos wt = U cosa, (6.154) 
is applied to the detector, then the instantaneous current through it becomes 
= JT o, (6.155) 


By averaging this current over one cycle of the ilternating voltage, we 
obtain the mean value of the current in the detector circuit: 


n a 
hay f ida =f f evmeda, (6.156) 
o o 
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We apply the formula (6.71) for the calculation of this integral and obtain the 
equation for the detector characteristic: 


Jay = Jo - blau). (6.157) 


Let the detector be connected to the output of a selective system under fluctua- 
ting input. Then the one-dimensional probability density of the voltage amplitude 
at the detector is by equation (6.39), and the constant component of the current 
in the detector circuit can be calculated with the aid of (5.20), in which we 


have to put y=—1: 


- vB 
Jy = J Jo: huU): Ze Pau. (6.158) 
(3 


We can compute this integral by representing the Bessel function /,(ul') as 


a power series (6.15) andthen, with the aid of formula (6.44), integrating by parts 
the thus obtained series in the integrand. Then simple transformations give : 


hy =) + (sy = he? ae (6.159) 
20 


i.e., a result coinciding with the expression (6.6). 


By analogous methods we can calculate moments of higher order for the mean 
current, and obtain, for instance, the formula (6.16). Of course, such a calculation 
method is, in principle, unsuitable for the analysis of the statistical properties of 
the instantaneous current ina detector circuit, as this current consists not only of 
the slow-changing mean current but also of rapidly changing high-frequency compo- 
nents. Inmanycases, however, these latter are of no interest. We shall make use 
of the same method for the investigation of the statistical properties of the mean 
current in a square-law detector, whose characteristic is given, as well known, 


by the equation 
Jay = h + oU. (6,160) 


Using formula (5.20) at v= | and assuming, as before, that the probability 
density w(U)is expressed by (6.38), we obtain: 


œ 


p 
Jay = f ntau emau. (6.161) 


o 
Now, the application of formula (6.44) gives 
Jay = h +H 2a0?, (6.162) 
We note that this result could have been obtained by averaging both sides of 
equation (6.160) and taking into account that in the case under consideration the 


mean square of the amplitude U is determined by the formula (6.45). 


By an analogous method it is simple to compute the mean square of the cur- 
rent Jay Setting in (5.20), v==2, we obtain: 


oo m 
J= fot aur Fe au. (6.163) 
i) 
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By representing the integral (6.163) as a sum of three integrals and by apply- 
ing to each of them formula (6.44), we obtain 


a}, = Ja + tado? + 80°". (6.164) 


The mean square of the noise component of the mear current is, by virtue of 
(6.162) and (6.164) 


on = 03, — Shy = 40s", (6.165) 


The mixed second moment of the mean current can be calculated, using ex- 
pression (5.24) and formula (6.75) for the two-dimensional probability density of the 
envelope. In this case one encounters somewhat greater computational difficulties 
then those above, 


The monograph of V.I. Bunimovich /4/ analyzes benide the problems consid- 
ered above, a number of other analogous problems. The case of a broken line 
characteristic of the nonlinear element under normally distributed input is 
examined in the work by I.N. Amiantov and V.I. Tikhonov /42/. 
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